Abstinence Education
Efforts to Assess the Accuracy and Effectiveness of Federally Funded Programs Gao ID: GAO-07-87 October 3, 2006Reducing the incidence of sexually transmitted diseases and unintended pregnancies is one objective of the Department of Health and Human Services (HHS). HHS provides funding to states and organizations that provide abstinence-until-marriage education as one approach to address this objective. GAO was asked to describe the oversight of federally funded abstinence-until-marriage education programs. GAO is reporting on (1) efforts by HHS and states to assess the scientific accuracy of materials used in these programs and (2) efforts by HHS, states, and researchers to assess the effectiveness of these programs. GAO reviewed documents and interviewed HHS officials in the Administration for Children and Families (ACF) and the Office of Population Affairs (OPA) that award grants for these programs.
Efforts by HHS and states to assess the scientific accuracy of materials used in abstinence-until-marriage education programs have been limited. This is because HHS's ACF--which awards grants to two programs that account for the largest portion of federal spending on abstinence-until-marriage education--does not review its grantees' education materials for scientific accuracy and does not require grantees of either program to review their own materials for scientific accuracy. In contrast, OPA does review the scientific accuracy of grantees' proposed educational materials. In addition, not all states that receive funding from ACF have chosen to review their program materials for scientific accuracy. In particular, 5 of the 10 states that GAO contacted conduct such reviews. Officials from these states reported using a variety of approaches in their reviews. While the extent to which federally funded abstinence-until-marriage education materials are inaccurate is not known, in the course of their reviews OPA and some states reported that they have found inaccuracies in abstinence-until-marriage education materials. For example, one state official described an instance in which abstinence-until-marriage materials incorrectly suggested that HIV can pass through condoms because the latex used in condoms is porous. HHS, states, and researchers have made a variety of efforts to assess the effectiveness of abstinence-until-marriage education programs; however, a number of factors limit the conclusions that can be drawn about the effectiveness of abstinence-until-marriage education programs. ACF and OPA have required their grantees to report on various outcomes that the agencies use to measure the effectiveness of grantees' abstinence-until-marriage education programs. In addition, 6 of the 10 states in GAO's review have worked with third-party evaluators to assess the effectiveness of abstinence-until-marriage education programs in their states. Several factors, however, limit the conclusions that can be drawn about the effectiveness of abstinence-until-marriage education programs. Most of the efforts to evaluate the effectiveness of abstinence-until-marriage education programs included in GAO's review have not met certain minimum scientific criteria--such as random assignment of participants and sufficient follow-up periods and sample sizes--that experts have concluded are necessary in order for assessments of program effectiveness to be scientifically valid, in part because such designs can be expensive and time-consuming to carry out. In addition, the results of efforts that meet the criteria of a scientifically valid assessment have varied and two key studies funded by HHS that meet these criteria have not yet been completed. When completed, these HHS-funded studies may add substantively to the body of research on the effectiveness of abstinence-until-marriage education programs.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-87, Abstinence Education: Efforts to Assess the Accuracy and Effectiveness of Federally Funded Programs
This is the accessible text file for GAO report number GAO-07-87
entitled 'Abstinence Education: Efforts to Assess the Accuracy and
Effectiveness of Federally Funded Programs' which was released on
November 16, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
October 2006:
Abstinence Education:
Efforts to Assess the Accuracy and Effectiveness of Federally Funded
Programs:
GAO-07-87:
GAO Highlights:
Highlights of GAO-07-87, a report to congressional requesters
Why GAO Did This Study:
Reducing the incidence of sexually transmitted diseases and unintended
pregnancies is one objective of the Department of Health and Human
Services (HHS). HHS provides funding to states and organizations that
provide abstinence-until-marriage education as one approach to address
this objective.
GAO was asked to describe the oversight of federally funded abstinence-
until-marriage education programs. GAO is reporting on (1) efforts by
HHS and states to assess the scientific accuracy of materials used in
these programs and (2) efforts by HHS, states, and researchers to
assess the effectiveness of these programs. GAO reviewed documents and
interviewed HHS officials in the Administration for Children and
Families (ACF) and the Office of Population Affairs (OPA) that award
grants for these programs.
What GAO Found:
Efforts by HHS and states to assess the scientific accuracy of
materials used in abstinence-until-marriage education programs have
been limited. This is because HHS‘s ACF”which awards grants to two
programs that account for the largest portion of federal spending on
abstinence-until-marriage education”does not review its grantees‘
education materials for scientific accuracy and does not require
grantees of either program to review their own materials for scientific
accuracy. In contrast, OPA does review the scientific accuracy of
grantees‘ proposed educational materials. In addition, not all states
that receive funding from ACF have chosen to review their program
materials for scientific accuracy. In particular, 5 of the 10 states
that GAO contacted conduct such reviews. Officials from these states
reported using a variety of approaches in their reviews. While the
extent to which federally funded abstinence-until-marriage education
materials are inaccurate is not known, in the course of their reviews
OPA and some states reported that they have found inaccuracies in
abstinence-until-marriage education materials. For example, one state
official described an instance in which abstinence-until-marriage
materials incorrectly suggested that HIV can pass through condoms
because the latex used in condoms is porous.
HHS, states, and researchers have made a variety of efforts to assess
the effectiveness of abstinence-until-marriage education programs;
however, a number of factors limit the conclusions that can be drawn
about the effectiveness of abstinence-until-marriage education
programs. ACF and OPA have required their grantees to report on various
outcomes that the agencies use to measure the effectiveness of
grantees‘ abstinence-until-marriage education programs. In addition, 6
of the 10 states in GAO‘s review have worked with third-party
evaluators to assess the effectiveness of abstinence-until-marriage
education programs in their states. Several factors, however, limit the
conclusions that can be drawn about the effectiveness of abstinence-
until-marriage education programs. Most of the efforts to evaluate the
effectiveness of abstinence-until-marriage education programs included
in GAO‘s review have not met certain minimum scientific criteria”such
as random assignment of participants and sufficient follow-up periods
and sample sizes”that experts have concluded are necessary in order for
assessments of program effectiveness to be scientifically valid, in
part because such designs can be expensive and time-consuming to carry
out. In addition, the results of efforts that meet the criteria of a
scientifically valid assessment have varied and two key studies funded
by HHS that meet these criteria have not yet been completed. When
completed, these HHS-funded studies may add substantively to the body
of research on the effectiveness of abstinence-until-marriage education
programs.
What GAO Recommends:
To address concerns about the scientific accuracy of materials used in
abstinence-until-marriage education programs, GAO recommends that the
Secretary of HHS develop procedures to help assure the accuracy of such
materials used in programs administered by ACF. HHS agreed to consider
this recommendation. HHS also provided information on steps it takes to
assure accuracy, which we have incorporated into the report, as
appropriate.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-87].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Marcia Crosse at (202)
512-7119 or crossem@gao.gov.
[End of Section]
Contents:
Letter:
Results in Brief:
Background:
Federal and State Efforts to Assess the Scientific Accuracy of
Materials Used in Abstinence-until-Marriage Education Programs Have
Been Limited:
A Variety of Efforts Have Been Made to Assess the Effectiveness of
Abstinence-until-Marriage Education Programs, but a Number of Factors
Limit the Conclusions That Can Be Drawn:
Conclusions:
Recommendation for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: HRSA's Technical Assistance Contract for Abstinence
Education:
Appendix II: Methodology for Identifying and Reviewing Research
Studies:
Appendix III: Comments from the Department of Health and Human
Services:
Appendix IV: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Definition of Abstinence Education:
Table 2: Funding Provided by HHS for the Three Main Abstinence-until-
Marriage Education Programs:
Table 3: ACF's Reporting Requirements for the State Program and the
Community-Based Program, Fiscal Year 2006:
Abbreviations:
ACF: Administration for Children and Families:
AFL: Adolescent Family Life:
ASPE: Office of the Assistant Secretary for Planning and Evaluation:
CDC: Centers for Disease Control and Prevention:
FAR: Federal Acquisition Regulation:
HHS: Department of Health and Human Services:
HIV: human immunodeficiency virus:
HPV: human papillomavirus:
HRSA: Health Resources and Services Administration:
NAC: National Abstinence Clearinghouse:
NIH: National Institutes of Health:
OMB: Office of Management and Budget:
OPA: Office of Population Affairs:
RFP: request for proposal:
STD: sexually transmitted disease:
TANF: Temporary Assistance for Needy Families:
United States Government Accountability Office:
Washington, DC 20548:
October 3, 2006:
Congressional Requesters:
Preventing sexually transmitted diseases (STD) and unintended
pregnancies among adolescents is an important public health challenge.
Although pregnancy and birth rates among female adolescents in the
United States have been declining since the early 1990s, the rates
continue to be high when compared with those in other industrialized
nations. The Centers for Disease Control and Prevention (CDC) reports
that about 141,000 children were born to girls 17 years old and younger
in the United States in 2003. CDC also reports that STDs
disproportionately affect adolescents, with adolescents and young
adults ages 15 to 24 acquiring almost half of the estimated 19 million
new infections each year.
Reducing the incidence of STDs and unintended pregnancies among
adolescents is an important objective for the Department of Health and
Human Services (HHS), which identifies as one of its goals the need to
reduce major threats to the health and well-being of Americans. Among
the efforts it supports to reduce the incidence of STDs and unintended
pregnancies among adolescents, HHS funds abstinence-until-marriage
education programs. Abstinence-until-marriage education programs, also
referred to as abstinence-only education programs, teach adolescents to
abstain from sexual activity until marriage in order to avoid risks of
unintended pregnancy, STDs, and related health problems.[Footnote 1]
The content of federally funded abstinence-until-marriage programs is
required to be consistent with several principles, such as teaching
that a mutually faithful monogamous relationship in the context of
marriage is the expected standard of human sexual activity, and that
abstinence from sexual activity is the only certain way to avoid STDs.
Abstinence-until-marriage education programs are delivered by a variety
of entities, including schools, human service agencies, faith-based
organizations, youth development groups, and pregnancy crisis centers.
Instructors can incorporate a variety of educational materials into
their abstinence-until-marriage education programs, including
textbooks, student manuals, brochures, slide presentations, and videos.
The three main federally funded abstinence-until-marriage programs are
the Abstinence Education Program (State Program), which is administered
by HHS's Administration for Children and Families (ACF); the Community-
Based Abstinence Education Program (Community-Based Program), which is
also administered by ACF; and the Adolescent Family Life (AFL) Program,
which is administered by HHS's Office of Population Affairs (OPA)
within the Office of Public Health and Science. Funding provided by HHS
for the three abstinence-until-marriage programs increased from about
$73 million in fiscal year 2001 to about $158 million in fiscal year
2005.
Recent studies have raised concerns about the accuracy of educational
materials used in abstinence-until-marriage education programs and
about the effectiveness of these programs. These studies have reported
that some of the materials used in abstinence-until-marriage education
programs contain, for example, scientifically inaccurate information
about anatomy and physiology as they relate to reproductive health as
well as misleading information about contraceptive failure rates and
STDs.[Footnote 2] State and federal agencies have also documented
inaccuracies in abstinence-until-marriage educational materials.
Further, studies examining the effectiveness of these programs have
reported varied results. For example, some researchers have reported
that abstinence-until-marriage education programs have resulted in
adolescents reporting less frequent sexual intercourse or fewer sexual
partners, while other researchers have reported that these types of
programs did not affect the frequency of sexual intercourse or were
ineffective in delaying the initiation of sexual intercourse.[Footnote
3]
You asked us to describe certain aspects of the oversight of federally
funded abstinence-until-marriage education programs. Our objectives
were to report on (1) efforts by HHS and states to assess the
scientific accuracy of materials used in abstinence-until-marriage
education programs and (2) efforts by HHS, states, and researchers to
assess the effectiveness of abstinence-until-marriage education
programs. You also asked us to describe how HHS selected a contractor
for the abstinence-until-marriage technical assistance contract that
was awarded in September 2002. This information is provided in appendix
I.
To describe the efforts by HHS and states to assess the scientific
accuracy of materials used in abstinence-until-marriage education
programs, we reviewed published reports, program announcements, Federal
Register notices, agency Web sites, and other documents related to
abstinence-until-marriage education. We focused our review on efforts
related to the three main federally funded abstinence-until-marriage
education programs administered by HHS, as well as efforts to review
the accuracy of scientific facts included in abstinence-until-marriage
education materials. We did not assess the criteria used to determine
the scientific accuracy of education materials or the quality of the
reviews. We interviewed officials from ACF, the Health Resources and
Services Administration (HRSA), OPA, and CDC. We also interviewed
officials from the 10 states that received the largest share of federal
funding (together accounting for 51 percent of the total funding in
fiscal year 2005) through the State Program for abstinence-until-
marriage education.[Footnote 4]
To describe efforts by HHS, states, and researchers to assess the
effectiveness of abstinence-until-marriage education programs, we
focused on efforts that examined the extent to which these programs
achieved their program goals. In general, these goals include teaching
adolescents to abstain from sexual activity until marriage in order to
avoid unintended pregnancies, STDs, and related health problems. As
part of our review, we compared these efforts to the design
characteristics that experts have identified as important for a
scientifically valid study of program effectiveness.[Footnote 5] We
reviewed journal articles and other published reports, agency budget
submissions, program announcements, agency and grantee performance
reports, Federal Register notices, agency Web sites, and other
documents related to abstinence-until-marriage education. (For a more
detailed description of our literature review methodology, see app.
II). We also interviewed officials from ACF, HRSA, OPA, CDC, the
National Institutes of Health (NIH), the Office of the Assistant
Secretary for Planning and Evaluation (ASPE), and 10 states that
received the largest share of federal funding for abstinence-only
education through the State Program in fiscal year 2005. We also
interviewed individuals from the National Campaign to Prevent Teen
Pregnancy, The Brookings Institution, ETR Associates, The Heritage
Foundation, and Advocates for Youth, and researchers from Case Western
Reserve University and Columbia University to obtain general
information regarding the state of the research on abstinence-until-
marriage education. We focused our review on efforts to assess the
scientific accuracy of materials and the effectiveness of the programs
during fiscal year 2006, and also reviewed the administration of the
programs back to fiscal year 2001. We also attended conferences
organized by ACF and OPA to learn about training that is provided to
grantees on scientific accuracy and program evaluations.
To describe how HHS selected a contractor for the abstinence-until-
marriage technical assistance contract that was awarded in September
2002, we reviewed the Request for Proposals and other related contract
documents. We also interviewed officials at HRSA, ACF, and the National
Abstinence Clearinghouse about the technical assistance contract. We
performed our work from October 2005 through September 2006 in
accordance with generally accepted government auditing standards.
Results in Brief:
Efforts by HHS and states to assess the scientific accuracy of
materials used in abstinence-until-marriage education programs have
been limited. This is because ACF--which awards grants through two
programs that account for the largest portion of federal spending on
abstinence-until-marriage education--does not review its grantees'
education materials for scientific accuracy and does not require
grantees of either program to review their own materials for scientific
accuracy. In addition, not all states that receive funding through
ACF's State Program have chosen to review their program materials for
scientific accuracy. In particular, 5 of the 10 states in our review
conduct such reviews. Officials from these states reported using a
variety of approaches in their reviews. In contrast, OPA does review
the scientific accuracy of AFL grantees' proposed educational materials
and any inaccuracies found must be corrected before the materials can
be used. While the extent to which federally funded abstinence-until-
marriage education materials are inaccurate is not known, in the course
of their reviews OPA and some states reported that they have found some
inaccuracies in abstinence-until-marriage education materials. For
example, OPA has required that a grantee correct several statements in
a true/false quiz--including statements about STDs and condom use--in
order for the quiz to be approved for use in its curriculum. In
addition, one state official described an instance in which abstinence-
until-marriage materials incorrectly suggested that HIV can pass
through condoms because the latex used in condoms is porous.
HHS, states, and researchers have made a variety of efforts to assess
the effectiveness of abstinence-until-marriage education programs;
however, a number of factors limit the conclusions that can be drawn
about the effectiveness of abstinence-until-marriage education
programs. To assess the effectiveness of their abstinence-until-
marriage education programs, ACF and OPA have required their grantees
to report on various outcomes that the agencies use to measure the
effectiveness of grantees' abstinence-until-marriage education
programs. For example, as of fiscal year 2006, states that receive
funding through the State Program are required to report annually on
four measures of the prevalence of adolescent sexual behavior in their
state, such as the rate of pregnancy among adolescents aged 15 to 17
years. To assess the effectiveness of both its State and Community-
Based Programs, ACF also analyzes trends in adolescent behavior, as
reflected in national data on birth rates among teens and the
proportion of surveyed high school students reporting that they have
had sexual intercourse. OPA requires grantees of the AFL Program to
develop and report on outcome measures that demonstrate the extent to
which grantees' programs are having an effect on program participants.
In addition, other HHS agencies and offices--ASPE, CDC and NIH--are
making efforts to assess the effectiveness of abstinence-until-marriage
education programs. Further, 6 of the 10 states in our review that
receive funding through the State Program have worked with third-party
evaluators to assess the effectiveness of abstinence-until-marriage
education programs in their states. Several factors, however, limit the
conclusions that can be drawn about the effectiveness of abstinence-
until-marriage education programs. Most of the efforts to evaluate the
effectiveness of abstinence-until-marriage education programs that we
describe in our review have not met certain minimum criteria--such as
random assignment of participants and sufficient follow-up periods and
sample sizes--that experts have concluded are necessary in order for
assessments of program effectiveness to be scientifically valid, in
part because such designs can be expensive and time-consuming to carry
out. In addition, the results of efforts that meet the criteria of a
scientifically valid assessment have varied and two key studies funded
by HHS that meet these criteria have not yet been completed. When
completed, these HHS-funded studies may add substantively to the body
of research on the effectiveness of abstinence-until-marriage education
programs.
To address concerns about the scientific accuracy of materials used in
abstinence-until-marriage education programs, we recommend that the
Secretary of HHS develop procedures to help assure the accuracy of such
materials used in the State and Community-Based Programs. To help
provide such assurance, the Secretary could consider alternatives such
as (1) extending the approach currently used by OPA to review the
scientific accuracy of the factual statements included in abstinence-
until-marriage education to materials used by grantees of ACF's
Community-Based Program and requiring grantees of ACF's State Program
to conduct such reviews or (2) requiring grantees of both programs to
sign written assurances in their grant applications that the materials
they propose using are accurate.
In commenting on a draft of this report, HHS agreed to consider
requiring grantees of both ACF programs to sign written assurances in
grant applications that the materials they use are accurate. In
addition, HHS noted that all federal grant applicants attest on a
standard form that information in their applications is correct.
However, it is not clear that this serves the purpose of assuring the
scientific accuracy of the educational materials. Further, the
curricula to be used are not required to be included with states'
applications. HHS's written comments also stated that ACF requires that
the Community-Based Program curricula conform to standards that are
grounded in scientific literature by requiring certain types of
information. However, the inclusion of certain types of information
does not necessarily ensure the accuracy of the scientific facts
included in the abstinence-until-marriage materials. In addition, HHS
noted in its written comments that we did not define the term
scientific accuracy and stated that it disagreed with certain findings
of the report because it was difficult to precisely determine the
criteria employed by GAO in making the recommendation as to scientific
accuracy. However, the objective of our work was to focus on efforts by
HHS and states to review the accuracy of scientific facts included in
abstinence-until-marriage education materials and not to perform an
independent assessment of the criteria used or the quality of the
reviews. With regard to effectiveness, HHS agreed that it may be too
soon to draw conclusions about the effectiveness of ACF's and OPA's
programs.
Background:
Recent statistics from CDC show that many high school students engage
in sexual behavior that places them at risk for unintended pregnancy
and STDs. In 2005, 46.8 percent of high school students reported that
they have ever had sexual intercourse, with 14.3 percent of students
reporting that they had had sexual intercourse with four or more
persons. The likelihood of ever having sexual intercourse varied by
grade, with the highest rate among 12th grade students (63.1 percent)
and the lowest rate among 9th grade students (34.3 percent). CDC also
has reported that the prevalence of certain STDs--including the rate of
chlamydia infection, the most frequently reported STD in the United
States--peaks in adolescence and young adulthood. According to CDC, in
2004 the chlamydia rates among adolescents 15 to 19 years old (1,579
cases per 100,000 adolescents) and young adults 20 to 24 years old
(1,660 cases per 100,000) were each more than twice the rates among all
other age groups.
HHS's current strategic plan includes the objectives to reduce the
incidence of STDs and unintended pregnancies and to promote family
formation and healthy marriages. These two objectives support HHS's
goals to reduce the major threats to the health and well-being of
Americans and to improve the stability and healthy development of
American children and youth, respectively. Abstinence-until-marriage
education programs are one of several types of programs that support
these objectives.[Footnote 6]
The three main federal abstinence-until-marriage education programs--
the State Program, the Community-Based Program, and the AFL Program--
provide grants to support the recipients' own efforts to provide
abstinence-until-marriage education at the local level.[Footnote 7]
These programs must comply with the statutory definition of abstinence
education (see table 1).[Footnote 8]
Table 1: Definition of Abstinence Education:
Abstinence education refers to an educational or motivational program
that:
A. has as its exclusive purpose, teaching the social, psychological,
and health gains to be realized by abstaining from sexual activity.
B. teaches abstinence from sexual activity outside marriage as the
expected standard for all school age children.
C. teaches that abstinence from sexual activity is the only certain way
to avoid out-of-wedlock pregnancy, sexually transmitted diseases, and
other associated health problems.
D. teaches that a mutually faithful monogamous relationship in context
of marriage is the expected standard of human sexual activity.
E. teaches that sexual activity outside of the context of marriage is
likely to have harmful psychological and physical effects.
F. teaches that bearing children out-of-wedlock is likely to have
harmful consequences for the child, the child's parents, and society.
G. teaches young people how to reject sexual advances and how alcohol
and drug use increases vulnerability to sexual advances; and.
H. teaches the importance of attaining self-sufficiency before engaging
in sexual activity.
Source: Social Security Act, § 510(b)(2) (codified at 42 U.S.C. §
710(b)(2)).
[End of table]
The State Program:
The State Program, administered by ACF, provides funding to its
grantees--states--for the provision of abstinence-until-marriage
education to those most likely to have children outside of
marriage.[Footnote 9] States that receive grants through the State
Program have discretion in how they use their funding to provide
abstinence-until-marriage education. Some require that organizations
apply for funds and use them to administer abstinence-until-marriage
education programs. Others may directly administer such programs. At
their discretion, states may also provide mentoring, counseling, and
adult supervision to adolescents to promote abstinence from sexual
activity until marriage.
The Personal Responsibility and Work Opportunity Reconciliation Act of
1996 established the State Program, and states were awarded grants
beginning in fiscal year 1998.[Footnote 10] Funds are allotted to each
state that submits the required annual application based on the ratio
of the number of low-income children in the state to the total number
of low-income children in all states. States are required to match
every $4 they receive in federal money with $3 of nonfederal money and
are required to report annually on the performance of the abstinence-
until-marriage education programs that they support or administer. In
fiscal year 2005, 47 states, the District of Columbia, and 3 insular
areas were awarded funding.[Footnote 11],[Footnote 12]
The Community-Based Program:
The Community-Based Program, which is also administered by ACF, is
focused on funding public and private entities that provide abstinence-
until-marriage education for adolescents from 12 to 18 years old, with
the purpose of creating an environment within communities that supports
adolescent decisions to postpone sexual activity until marriage. The
Community-Based Program provides grants for school-based programs,
adult and peer mentoring, and parent education groups. The Community-
Based Program first awarded grants in fiscal year 2001.[Footnote 13]
Grantees of the Community-Based Program are selected through a
competitive process and are evaluated according to several criteria,
such as the extent to which they have demonstrated that a need exists
for abstinence-until-marriage education for a targeted population or in
a specific geographic location. Grantees are required to report to ACF,
on a semiannual basis, on the performance of their programs. For fiscal
year 2005, 63 grants were awarded to organizations and other
entities.[Footnote 14]
The AFL Program:
The AFL Program supports programs that provide abstinence-until-
marriage education.[Footnote 15] The primary purpose of these programs
is to find effective means of reaching preadolescents and adolescents
before they become sexually active in order to encourage them to
abstain from sexual activity and other risky behaviors. Under the AFL
Program, OPA awards competitive grants to public or private nonprofit
organizations or agencies, including community-based and faith-based
organizations, to facilitate abstinence-until-marriage education in a
variety of settings, including schools and community centers.
Established in 1981, the AFL Program began awarding grants in fiscal
year 1982. AFL Program grantees include school districts, youth
development groups, and medical centers. Grant applicants are evaluated
based on several criteria, such as the extent to which they provide a
clear statement of mission, goals, measurable objectives, and a
reasonable method for achieving their objectives. Grantees are required
to conduct evaluations of certain aspects of their programs and report
annually on their performance. As of August 2006, OPA funded 58
abstinence-until-marriage education programs, and most of these were
focused on reaching young adolescents from the ages of 9 to 14.
Funding for Abstinence-until-Marriage Education Has Increased:
Funding provided by HHS for abstinence-until-marriage education
programs has increased steadily since 2001 (see table 2). For the three
main programs combined--the State Program, the Community-Based Program,
and the AFL Program--the amount of agency funding increased from about
$73 million in fiscal year 2001 to about $158 million in fiscal year
2005. Nearly all of this increase was for the Community-Based program;
funding under this program increased by about $84 million from fiscal
years 2001 through 2005. In fiscal year 2005, agency funding for the
Community-Based Program constituted the largest share of the total
funding (about 66 percent) for the three main programs combined.
Table 2: Funding Provided by HHS for the Three Main Abstinence-until-
Marriage Education Programs:
Dollars in millions.
Fiscal year: 2001;
State Program[A,B]: $43;
Community-Based Program: $20;
AFL Program: $10.
Fiscal year: 2002;
State Program[A,B]: 43;
Community-Based Program: 40;
AFL Program: 12.
Fiscal year: 2003;
State Program[A,B]: 44;
Community-Based Program: 55;
AFL Program: 13.
Fiscal year: 2004;
State Program[A,B]: 41;
Community-Based Program: 75;
AFL Program: 13.
Fiscal year: 2005;
State Program[A,B]: 41;
Community-Based Program: 104;
AFL Program: 13.
Sources: ACF, OPA, and HRSA.
Notes: Figures are rounded to nearest $1,000,000. Funding levels
represent the total amount of grants awarded and funding for program-
related support, such as technical assistance and evaluation.
[A] States that receive funding are required to match every $4 they
receive of federal funds with $3 of nonfederal money.
[B] The amount of funding provided by HHS for the State Program has
generally varied by year because the states that have applied for
funding each year have varied.
[End of table]
Within each of the three main abstinence-until-marriage education
programs, the amount of individual grants varied.[Footnote 16] In
fiscal year 2005, the State Program's annual grants ranged from $57,057
to $4,777,916 and the median annual grant amount was $569,675. That
same year, the Community-Based Program's annual grants ranged from
$213,276 to $800,000 and the median grant amount was $642,250. In
fiscal year 2006, the AFL Program's annual grants ranged from $95,676
to $300,000 and the median grant amount was $225,000.
Federal Agency Responsibilities Related to Abstinence-until-Marriage
Education:
Five organizational units located within HHS--ACF, OPA, CDC, ASPE, and
NIH--have responsibilities related to abstinence-until-marriage
education. ACF and OPA administer the three main federal abstinence-
until-marriage education programs. CDC supports abstinence-until-
marriage education at the national, state, and local levels. CDC, ASPE,
and NIH are sponsoring research on the effectiveness of abstinence-
until-marriage programs.
ACF:
ACF is responsible for federal programs that promote the economic and
social well-being of families, children, individuals, and communities.
ACF administers and provides oversight of both the State Program and
the Community-Based Program by, among other things, awarding grants,
providing training and technical assistance to grantees, and requiring
annual performance reporting from grantees. ACF has been responsible
for the State Program since June 2004 and the Community-Based Program
since October 2005. HRSA previously administered these programs.
OPA:
OPA has responsibility for advising the Secretary of HHS on a wide
range of reproductive health topics, including adolescent pregnancy and
family planning. The office is also responsible for administering
programs that provide services for pregnant and parenting teens and
prevention programs, such as abstinence-until-marriage education
programs. OPA administers and provides oversight of the AFL Program by
awarding grants, providing training and technical assistance to
grantees, and requiring annual performance reporting from grantees.
CDC:
CDC is primarily responsible for the prevention and control of
infectious and chronic diseases, including STDs. CDC provides funding
to state and local education agencies in their efforts to support
comprehensive school health education and HIV/STD prevention education
programs, and CDC officials told us that some of these are focused on
abstinence. CDC also provides funding to several state education
agencies to implement various abstinence projects, such as
collaboration-building among agencies to increase the impact of their
efforts to encourage abstinence. Further, CDC develops tools to assist
state and local education agencies with their health education
programs. CDC provides funding to several national organizations to
build the capacity of abstinence-until-marriage education providers.
Organizations' activities include, but are not limited to, the
development and distribution of educational materials. CDC is also
sponsoring research on the effectiveness of an abstinence-until-
marriage education program.
ASPE:
ASPE advises the Secretary of HHS in several areas, including policy
development in health, human services, data, and science. ASPE is
responsible for the development of policy analyses and it conducts
research and evaluation studies in several areas, including the health
of children and adolescents. ASPE is currently sponsoring research on
the effectiveness of abstinence-until-marriage education programs.
NIH:
NIH is the primary federal agency that conducts and supports medical
and behavioral research among various populations, including children
and adolescents. NIH is currently sponsoring research on the
effectiveness of abstinence-until-marriage education programs.
Federal and State Efforts to Assess the Scientific Accuracy of
Materials Used in Abstinence-until-Marriage Education Programs Have
Been Limited:
Efforts by HHS and states to assess the scientific accuracy of
materials used in abstinence-until-marriage education programs have
been limited. ACF--which awards grants to two programs that account for
the largest portion of federal spending on abstinence-until-marriage
education--does not review its grantees' education materials for
scientific accuracy and does not require grantees of either program to
review their own materials for scientific accuracy. In addition, not
all states funded through the State Program have chosen to review their
program materials for scientific accuracy. In contrast to ACF, OPA has
reviewed the scientific accuracy of grantees' proposed educational
materials and corrected inaccuracies in these materials.
ACF Does Not Review Program Materials for Scientific Accuracy and Does
Not Require Grantees to Do So, though Some State Grantees Have
Conducted Such Reviews:
There have been limited efforts to review the scientific accuracy of
educational materials used in ACF's State and Community-Based Programs-
-the two programs that account for the largest portion of federal
spending on abstinence education. ACF does not review materials for
scientific accuracy in either reviewing grant applications[Footnote 17]
or in overseeing grantees' performance. Prior to fiscal year 2006,
State Program and Community-Based Program applicants were not required
to submit copies of their proposed educational materials with their
applications. While ACF required grantees of the Community-Based
Program--but not the State Program--to submit their educational
materials with their fiscal year 2006 applications, ACF officials told
us that grantee applications and materials are only reviewed to ensure
that they address all aspects of the scope of the Community-Based
Program, such as the A-H definition of abstinence education.[Footnote
18] Further, documents provided to us by ACF indicate that the agency
does not review grantees' educational materials for scientific accuracy
as a routine part of its oversight activities. In addition, ACF also
does not require its grantees to review their own materials for
scientific accuracy. Similarly, when HRSA was responsible for the State
and Community-Based Programs, the agency did not review materials used
by grantees for scientific accuracy or require grantees to review their
own materials.
Not all grantees of the State Program have chosen to review the
scientific accuracy of their educational materials. Officials from 5 of
the 10 states in our review reported that their states have chosen to
conduct such reviews.[Footnote 19],[Footnote 20] Officials in these
states identified a variety of reasons why their states reviewed
abstinence-until-marriage educational materials, including program
requirements, state education laws and guidelines, and past lawsuits,
to ensure that materials used in abstinence-until-marriage programs
were accurate. For example, Michigan's Revised School Code states that
materials and instruction in the sex education curricula, including
information on abstinence, "shall not be medically
inaccurate,"[Footnote 21] and Ohio's fiscal year 2007 abstinence-until-
marriage education program guidance states that abstinence-until-
marriage educational materials "should be medically accurate in all
assertions."[Footnote 22]
The five states we contacted that review abstinence-until-marriage
educational materials for scientific accuracy have used a variety of
approaches in their reviews. Some states contracted with medical
professionals--such as nurses, gynecologists, and pediatricians--to
serve as medical advisors who review program materials and use their
expertise to determine what is and is not scientifically accurate. Some
states have created checklists or worksheets to guide their staff
conducting the review and document findings of inaccuracy or
verification of a statement. All five states use medical professionals
in conducting these reviews. One of the states requires that all
statistics or scientific statements cited in a program's materials are
sourced to CDC or a peer-reviewed medical journal. Officials from this
state told us that if statements in these materials cannot be
attributed to these sources, the statements are required to be removed
until citations are provided and materials are approved. Officials from
this state told us they have also supplemented their review of program
materials with on-site classroom observations to assess the scientific
accuracy of the information presented to students.
Officials from two of the five states reported that they have found
inaccuracies as a result of their reviews. For example, one state
official stated that because information is constantly evolving, state
officials have had to correct out-of-date scientific information. In
addition, this official cited an instance where materials incorrectly
suggested that HIV can pass through condoms because the latex used in
condoms is porous. In addition, this official provided documentation
that the state has had to correct a statement indicating that when a
person is infected with the human papillomavirus,[Footnote 23] the
virus is "present for life" because, in almost all cases, this is not
true. State officials who have identified inaccuracies told us that
they informed their grantees of inaccuracies so that they could make
corrections in their individual programs. One state official added that
she contacted the authors of the materials to report an inaccuracy.
Some of the educational materials that states have reviewed are
materials that are commonly used in the Community-Based Program.
Officials from four of the five states that review materials for
scientific accuracy told us that they have each reviewed at least one
of the five curricula most commonly used in the Community-Based Program
because programs in their state were using them: Choosing the Best,
WAIT Training, Sex Can Wait, A.C. Green's Game Plan Abstinence Program,
and Worth the Wait. Based on ACF documents, we found that there were 58
different curricula used by grantees of the Community-Based Program in
fiscal year 2005. However, more than half of the grantees of the
Community-Based Program reported using at least one of these five
curricula.[Footnote 24]
While there has been limited review of materials used in the State and
Community-Based Programs, grantees of these programs have received some
technical assistance designed to improve the scientific accuracy of
their materials. For example, ACF officials reported that the agency
provided a conference for grantees of the Community-Based Program in
February 2006 that included a presentation focused on medical accuracy,
including a discussion of state legislative proposals that would
require medical accuracy in abstinence-until-marriage education, and
how to identify reliable data. In addition, in 2002, HRSA awarded a
contract to the National Abstinence Clearinghouse requiring, among
other things, that the contractor develop and implement a program to
provide medically accurate information and training to grantees of the
State and Community-Based Programs.[Footnote 25] (See app. I for a
description of HRSA's process for awarding this contract). The portion
of the contract that focused on providing medically accurate
information to grantees was subcontracted to the Medical Institute for
Sexual Health (Medical Institute),[Footnote 26] which has conducted
presentations at regional educational conferences to provide grantees
with medical and scientific information, such as updated information on
condoms and STD transmission. The Medical Institute has also provided
consultative services to grantees by responding to medical and
scientific questions.
OPA Reviews Materials Used by AFL Program Grantees for Scientific
Accuracy:
In contrast to ACF, OPA reviews for scientific accuracy the educational
materials used by AFL Program grantees. Specifically, OPA reviews its
grantees' proposed educational materials for scientific accuracy before
they are used. Agency officials stated that they began to review these
materials while litigation concerning the AFL Program was ongoing. OPA
continued to review these materials as part of a 1993 settlement to
this lawsuit.[Footnote 27] The settlement agreement expired in 1998,
though the agency has continued to review grantees' proposed
educational materials for accuracy as a matter of policy. OPA officials
told us that grant applicants submit summaries of materials they
propose to use, though the materials are not reviewed for scientific
accuracy until after grantees have been selected. OPA officials said
that after grants are awarded, a medical education specialist (in
consultation with several part-time medical experts) reviews the
grantees' printed materials and other educational media, such as
videos. OPA officials explained that the medical education specialist
must approve all materials before they are used. On many occasions, OPA
grantees have proposed using--and therefore OPA has reviewed--materials
commonly used in the Community-Based Program. For example, an OPA
official told us that the agency had reviewed three of the Community-
Based Program's commonly used curricula--Choosing the Best, Sex Can
Wait, and A.C. Green's Game Plan Abstinence Program--and is also
currently reviewing another curriculum commonly used by Community-Based
Program grantees, WAIT Training.[Footnote 28]
OPA officials stated that the medical education specialist has
occasionally found and addressed inaccuracies in grantees' proposed
educational materials. OPA officials stated that these inaccuracies are
often the result of information being out of date because, for example,
medical and statistical information on STDs changes frequently. OPA has
addressed these inaccuracies by either not approving the materials in
which they appeared or correcting the materials through discussions
with the grantees and, in some cases, the authors of the materials. In
fiscal year 2005, OPA disapproved of a grantee using a specific
pamphlet about STDs because the pamphlet contained statements about STD
prevention and HIV transmission that were considered incomplete or
inaccurate. For example, the pamphlet stated that there was no cure for
hepatitis B, but the medical education specialist required the grantee
to add that there was a preventive vaccine for hepatitis B. In
addition, OPA required that a grantee correct several statements in a
true/false quiz--including statements about STDs and condom use--in
order for the quiz to be approved for use. For example, the medical
education specialist changed a sentence from "The only 100% effective
way of avoiding STDs or unwanted pregnancies is to not have sexual
intercourse." to "The only 100% effective way of avoiding STDs or
unwanted pregnancies is to not have sexual intercourse and engage in
other risky behaviors."
While OPA and some states have reviewed their grantees' abstinence-
until-marriage education materials for scientific accuracy,[Footnote
29] these types of reviews have the potential to affect abstinence-
until-marriage education providers more broadly. Such efforts may
create an incentive for authors of abstinence-until-marriage education
materials to ensure they are accurate. Thus, some authors of abstinence-
until-marriage education materials have recently updated materials in
their curricula following reports that questioned their accuracy. For
example, one of the most widely used curricula used by grantees of the
Community-Based Program--WAIT Training--has been recently updated and
provides the updated information on its Web site. A representative from
WAIT Training stated that the company recently revised its curriculum,
in part, in response to a congressional review that found inaccuracies
in its abstinence-until-marriage education materials.
A Variety of Efforts Have Been Made to Assess the Effectiveness of
Abstinence-until-Marriage Education Programs, but a Number of Factors
Limit the Conclusions That Can Be Drawn:
HHS, states, and researchers have made a variety of efforts to assess
the effectiveness of abstinence-until-marriage education programs;
however, a number of factors limit the conclusions that can be drawn
about the effectiveness of these programs. ACF and OPA have required
their grantees to report on various outcomes used to measure the
effectiveness of grantees' abstinence-until-marriage education
programs,[Footnote 30] though the reporting requirements for each of
the three abstinence-until-marriage programs differ. In addition, to
assess the effectiveness of the State and Community-Based Programs, ACF
has analyzed national data on adolescent birth rates and the proportion
of adolescents who report having had sexual intercourse. Other
organizational units within HHS--ASPE, CDC, and NIH--are funding
studies designed to assess the effectiveness of abstinence-until-
marriage education programs in delaying sexual initiation, reducing
pregnancy and STD rates, and reducing the frequency of sexual activity.
Despite these efforts, several factors limit the conclusions that can
be drawn about the effectiveness of abstinence-until-marriage education
programs. Most of the efforts to evaluate the effectiveness of
abstinence-until-marriage education programs that we describe in our
review have not met certain minimum criteria that experts have
concluded are necessary in order for assessments of program
effectiveness to be scientifically valid, in part because such designs
can be expensive and time-consuming to carry out. In addition, the
results of some efforts that meet the criteria of a scientifically
valid assessment have varied, and two key studies that meet these
criteria have not yet been completed.
HHS, States, and Researchers Have Made a Variety of Efforts to Assess
the Effectiveness of Abstinence-until-Marriage Education Programs:
Efforts of HHS, states, and researchers to assess the effectiveness of
abstinence-until-marriage education programs have included ACF and OPA
requiring grantees to report data on outcomes of their abstinence-
until-marriage education programs; ACF analyzing national data on
adolescent behavior and birth rates; and other HHS agencies, states,
and researchers funding or conducting studies to assess the
effectiveness of abstinence-until-marriage education programs.
ACF and OPA Have Required Grantee Reporting of Data on Outcomes:
ACF has made efforts to assess the effectiveness of abstinence-until-
marriage education programs funded by the State Program and the
Community-Based Program. One of ACF's efforts has been to require
grantees of both programs to report data on outcomes, though the two
programs have different requirements for the outcomes grantees must
report. For the State Program, as of fiscal year 2006, grantees must
report annually on four measures of the prevalence of adolescent sexual
behavior in their states, such as the rate of pregnancy among
adolescents aged 15 to 17 years, and compare these data to program
targets over 5 years. To report on these four measures, states may
choose the data sources they will use.[Footnote 31] States must also
develop and report on two additional performance measures that are
related to the goals of their programs.[Footnote 32] (See table 3 for a
list of ACF's fiscal year 2006 reporting requirements for the State
Program.)
Table 3: ACF's Reporting Requirements for the State Program and the
Community-Based Program, Fiscal Year 2006:
State Program:
* Report on four performance measures: (1) rate of pregnancy among
female teenagers aged 15 to 17, (2) proportion of adolescents who have
engaged in sexual intercourse, (3) incidence of youths 15 to 19 years
old who have contracted selected STDs, and (4) rate of births among
female teenagers aged 15 to 17;
Community-Based Program:
* Report on program goals that are developed by grantees with a third-
party evaluator. Such outcomes could include changes in knowledge about
abstinence or declared behavior among participants of abstinence-until-
marriage programs.
State Program:
* Develop and report on two additional performance measures that are
related to individual program goals. Past examples of these additional
measures have included the percentage of high school students who
reported using drugs or alcohol before intercourse and the percentage
of high school students who had sexual intercourse for the first time
before age 13;
Community-Based Program:
* Report on program "outputs" the number of youth served, the hours of
service provided to each youth, and the number of youths who complete
the program. Grantees choose additional outputs that allow for
effective monitoring and management of the project. The additional
outputs may include tracking the number of staff trained to provide
services, the number of events hosted, number of marketing materials
distributed, and so forth.
Sources: State and Community-Based Programs' announcements, fiscal year
2006.
[End of table]
As of fiscal year 2006, ACF requires Community-Based Program grantees
to develop and report on outcome measures designed to demonstrate the
extent to which grantees' community-based abstinence education programs
are accomplishing their program goals.[Footnote 33] ACF requires
grantees of the Community-Based Program to contract with third-party
evaluators, who are responsible for both helping grantees develop the
outcome measures and monitoring grantee performance against the
measures,[Footnote 34] but because this is a new requirement
established for fiscal year 2006 grantees, ACF has not yet received the
results of these evaluations. In addition to outcome reporting, ACF
requires grantees of the Community-Based Program to report on program
"outputs," which measure the quantity of program activities and other
deliverables, such as the number of participants who are served by the
abstinence-until-marriage education programs. According to ACF
officials, the agency requires grantees of both the State Program and
the Community-Based Program to report on program outcomes in order to
monitor grantees' performance, target training, and technical
assistance, and help grantees improve service delivery. (See table 3
for a list of ACF's fiscal year 2006 reporting requirements for the
Community-Based Program.)
ACF's fiscal year 2006 reporting requirements for grantees of the State
Program are the same as HRSA's when it administered the State Program.
In contrast, ACF's fiscal year 2006 reporting requirements for the
Community-Based Program differ from HRSA's reporting requirements for
the program. For example, for Community-Based Program grants awarded in
fiscal year 2001, HRSA required[Footnote 35] grantees to report on the
effectiveness of their programs, as measured by program participation
as well as behavioral and biological outcomes.[Footnote 36] These
performance measures were modified for fiscal year 2002, in part HHS
officials explained, because of concerns expressed by members of the
abstinence-education community that the original performance measures
did not accurately reflect the efforts of the grantees of the Community-
Based Program. For grants awarded from fiscal years 2002 through 2004,
HRSA required grantees of the Community-Based Program to report on a
combination of program outputs, such as the proportion of adolescents
who completed an abstinence-until-marriage education program, and
measures of adolescent intentions, such as the proportion of
adolescents who committed to abstaining from sexual activity until
marriage.[Footnote 37] For grants awarded in fiscal year 2005, when ACF
assumed responsibility for the Community-Based Program from HRSA,
grantees were not required to report on any specific performance
measures.
OPA has also made efforts to assess the effectiveness of the AFL
Program. Specifically, OPA requires grantees of the AFL Program to
develop and report on outcome measures that are used to help
demonstrate the extent to which grantees' programs are having an effect
on program participants.[Footnote 38],[Footnote 39] According to OPA
officials, the agency recommends that grantees report on outcome
measures, such as participants' knowledge of the benefits of abstinence
and their reported intentions to abstain from sexually activity,
reported beliefs in their ability to remain abstinent, and reported
parental involvement in their lives. To collect data on these outcome
measures and any others, OPA requires all grantees funded in fiscal
year 2004 and beyond to administer, at a minimum, a standardized
questionnaire--developed by OPA--to their program participants, both
when participants begin an abstinence-only education program and after
the program's completion.[Footnote 40] The standardized questionnaire
includes questions intended to obtain information on participants'
reported involvement in extracurricular activities, behaviors linked to
health risks, attitudes and intentions about abstinence, and opinions
about the consequences of premarital sexual activity. Like ACF, OPA
requires its grantees to contract with independent evaluators, such as
colleges or universities, which are responsible for evaluating the
effectiveness of grantees' individual abstinence-until-marriage
education programs.[Footnote 41] In addition to evaluating the extent
to which grantees are meeting their goals, OPA officials stated that
the independent evaluators may also provide input to grantees of the
AFL Program on other aspects of the program to improve their service
delivery. Unlike ACF, OPA requires that the third-party evaluations
incorporate specific methodological characteristics, such as control
groups or comparison groups[Footnote 42] and sufficient sample
sizes.[Footnote 43] In addition, OPA requires that the evaluations for
grantees funded in fiscal year 2004 and beyond account for baseline and
follow-up data obtained from the standardized questionnaires.
OPA's requirement that grantees use a standardized set of
questionnaires, with data from these questionnaires used in
evaluations, differs from OPA's previous requirements. Previously,
grantees of the AFL Program were not required to use a standardized
method for collecting data that could be used to assess the
effectiveness of their programs; instead, grantees chose their own data
collection instruments. As a result, an OPA official explained, the
collected data varied from one project to another. OPA officials said
that the agency developed the standardized questionnaire to ensure
uniformity in the data collected and allow the agency to more
effectively aggregate the data reported in evaluations of individual
abstinence-until-marriage education programs.
OPA officials told us that they plan to aggregate information from
certain questions in the standardized set of questionnaires in order to
report on certain performance measures as part of the agency's annual
performance reports.[Footnote 44] The measures include the extent of
parental involvement in adolescents' lives and the extent to which
adolescents understand the benefits of abstinence. An agency official
stated that the agency expects to begin receiving data from grantees
that are using these questionnaires in January 2007. OPA did not
previously have long-term measures of the performance of the AFL
Program. Its current measures were developed in collaboration with the
Office of Management and Budget (OMB) in response to an OMB review in
2004 that found that the AFL Program did not have any annual
performance measures for measuring progress toward long-term goals.
In addition to requiring their grantees to report on outcomes used to
assess program effectiveness, both ACF and OPA have provided technical
assistance and training to their grantees in order to support grantees'
own program evaluation efforts. For example, in November 2005 the two
agencies sponsored an evaluation conference for abstinence-until-
marriage grantees that included presentations about evaluations and
their methodology. Similarly, ACF's Office of Planning, Research, and
Evaluation sponsors annual evaluation conferences, and an ACF official
told us that a recent conference placed "a significant emphasis" on the
evaluation of abstinence-until-marriage education programs. In
addition, HHS officials told us that ACF, along with ASPE, is funding a
multiyear project that is designed to identify gaps in abstinence
education evaluation and technical assistance needs, develop materials
on abstinence education evaluation, deliver technical assistance and
capacity-building activities related to program evaluation, and develop
research reports related to abstinence education. OPA officials also
told us that they attempt to help ensure grantees' progress and
effectiveness by offering various technical assistance workshops and
conferences. For example, in May 2006 OPA provided a 2-day training
conference to its grantees on the importance of program evaluations and
administering evaluation instruments. In addition, OPA officials stated
that the agency contracts with evaluation consultants, who review
grantees' evaluation tools and activities. OPA officials explained that
these consultants provide in-depth technical assistance to grantees on
how to improve grantees' evaluations.
ACF Uses National Data on Adolescent Behavior as a Measure of the
Overall Effectiveness of Its Two Programs:
Requiring outcome reporting from state and community-based grantees is
not ACF's only effort to assess the effectiveness of its two programs.
ACF also analyzes trends in adolescent behavior, as reflected in
national data on birth rates among teens and the proportion of surveyed
high school students reporting that they have had sexual
intercourse.[Footnote 45] ACF uses these national data as a measure of
the overall effectiveness of its State and Community-Based Programs,
comparing the national data to program targets. In its annual
performance reports, the agency summarizes the progress being made
toward lowering the rate of births to unmarried teenage girls and the
proportion of students (grades 9-12) who report having ever had sexual
intercourse.
ACF's use of national data to assess the effectiveness of the State and
Community-Based Programs represents a change from how HRSA assessed the
overall effectiveness of these programs. Whereas ACF compares national
data on adolescent behavior to program targets, HRSA aggregated data
from its state and community-based grantees. HRSA's state grantees were
allowed to select the data sources used to gauge their progress against
certain performance measures. For example, in its annual performance
reports on the State Program, HRSA reported information on the
percentage of its state grantees meeting target rates for reducing the
proportion of adolescents who have engaged in sexual intercourse, the
incidence of youths aged 15 to 19 who have contracted selected STDs,
and the rate of births among youths aged 15 to 17. To determine their
progress in meeting their target rates, some state grantees, for
example, reported national data from the Youth Risk Behavior
Surveillance System, while other grantees reported state-collected
data. After ACF assumed responsibility for the State and Community-
Based Programs from HRSA, ACF began using national data on adolescent
behavior as a measure of the programs' effectiveness. According to ACF
officials, the agency changed how it assessed its programs out of
concern over the quality of the data state grantees were using in their
performance reporting and because the agency wanted to use parallel
measures of effectiveness for both programs. For example, according to
state performance reports for fiscal year 2001 that we reviewed, two
reports did not include adolescent pregnancy rates that year because
the states did not collect data on abortions among this
population.[Footnote 46] In addition, ACF officials told us that they
decided not to use national data on STDs as a measure of program
effectiveness because the goal of reducing STD rates is not as central
to the State and Community-Based Programs as reducing sexual activity
and birth rates among teens. However, one official stated that reducing
STDs is an important "by-product" of the programs.
Some States Have Made Additional Efforts to Assess the Effectiveness of
Abstinence-until-Marriage Education Programs:
Some states have made additional efforts to assess the effectiveness of
abstinence-until-marriage education programs, although they are not
required by ACF.[Footnote 47] Specifically, we found that 6 of the 10
states in our review that receive funding through ACF's State Program
have made efforts to conduct evaluations of selected abstinence-until-
marriage programs in their state. All 6 of the states worked with third-
party evaluators, such as university researchers or private research
firms, to perform the evaluations, which in general measure self-
reported changes in program participants' behavior and attitudes
related to sex and abstinence as indicators of program effectiveness.
To obtain this information, the third-party evaluators have typically
relied on surveys administered to program participants at the start of
a program, its conclusion, and during a follow-up period anywhere from
3 months to almost 3 years after the conclusion. The third-party
evaluations for 4 of the 6 states in our review have been completed as
of February 2006, and the results of these studies have
varied.[Footnote 48]
Among the 4 states that have completed third-party evaluations, 3
states require the abstinence programs in their state to measure
reported changes in participants' behavior as an indicator of program
effectiveness--both at the start of the program and after its
completion. The 3 states require their programs to track participants'
reported incidence of sexual intercourse. In addition, 2 states require
their programs to track biological outcomes, such as pregnancies,
births, or STDs. In addition, 6 of the 10 states in our review require
their programs to track participants' attitudes about abstinence and
sex, such as the number of participants who make pledges to remain
abstinent.
Some states also provide technical assistance to the abstinence-until-
marriage programs they support in their state. This assistance is
designed to help programs evaluate and improve their effectiveness.
Officials from 5 of the 10 states in our review either told us or
provided documentation that they provide technical assistance on
evaluations to abstinence programs in their state. One state official
said that the abstinence-until-marriage programs supported by the state
were found to be ill-prepared to conduct evaluations themselves, and
that she now requires these programs to dedicate a portion of their
grants to contract with a third-party or state evaluator to assist them
in program-level evaluations. Officials from another state told us that
they contract with a private organization of public health
professionals in order to provide evaluation consultation and technical
assistance for the abstinence-until-marriage programs the state
supports.
ASPE, CDC, and NIH Are Funding Studies Designed to Assess the
Effectiveness of Abstinence-until-Marriage Education Programs:
In addition to ACF and OPA, other organizational units within HHS have
made efforts to assess the effectiveness of abstinence-until-marriage
education programs. ASPE is currently sponsoring a study of the
Community-Based Program and a study of the State Program.[Footnote 49]
For the former program, ASPE has contracted with Abt Associates to help
design the study, and an ASPE official told us that once the agency
selects an appropriate design, it will competitively award a contract
to conduct the study.[Footnote 50] For the latter program, ASPE has
contracted with Mathematica Policy Research, Inc. (Mathematica), which
is in the process of examining the impact of five programs funded
through the State Program on participants' attitudes and behaviors
related to abstinence and sex.[Footnote 51] As of August 2006,
Mathematica has published two reports on findings from its study--an
interim report documenting the experiences of schools and communities
that receive abstinence-until-marriage education funding, and a report
on the first-year impacts of selected state abstinence-until-marriage
education programs.[Footnote 52],[Footnote 53] Mathematica's final
report, which has not been completed, will examine the impact of the
State Program on behavioral outcomes, including abstinence, sexual
activity, risk of STDs, risk of pregnancy, and drug and alcohol
use.[Footnote 54] An ASPE official told us that the agency expects a
final report to be published in 2007.
Like ASPE, CDC has made its own effort to assess the effectiveness of
abstinence-until-marriage education. CDC is sponsoring a study to
evaluate the effectiveness of two middle school curricula--one that
complies with abstinence education program requirements and one that
teaches a combination of abstinence and contraceptive information and
skills.[Footnote 55] In CDC's study, five middle schools chosen at
random will receive a program consisting of abstinence-until-marriage
education exclusively; five schools will receive comprehensive sex
education, which also includes information on contraception; and five
schools will be assigned to a control group. The study will examine the
relative effectiveness of the programs on behavioral outcomes such as
reported sexual risk behaviors and changes in attitudes related to
abstinence and sex. CDC plans to recruit approximately 1,500 seventh
grade students into its study and will follow them over a 2-year
period. The agency expects to complete the study in 2009.
NIH has funded studies comparing the effectiveness of education
programs that focus only on abstinence with the effectiveness of sex
education programs that teach both abstinence and information about
contraception. As of August 2006, NIH is funding five studies, which in
general are comparing the effects of these two types of programs on the
sexual behavior and related attitudes among groups of either middle
school or high school students. For example, in one NIH study,
researchers are using groups of seventh and eighth grade adolescents to
assess the impact of a variety of programs on, among other issues,
adolescents' reported sexual activities, knowledge, and beliefs. For
this study, researchers are comparing these outcomes among students who
received abstinence-until-marriage education; students who received a
combination of abstinence and contraceptive education; and students who
participated in a general health class, who serve as a comparison
group. NIH expects both this study and its other four studies to be
competed in 2006.
Other Researchers Have Also Made Efforts to Assess the Effectiveness of
Abstinence-until-Marriage Education Programs:
In addition to the efforts of researchers working on behalf of HHS and
states, other researchers--such as those affiliated with universities
and various advocacy groups--have made efforts to study the
effectiveness of abstinence-until-marriage education programs. This
work includes studies of the outcomes of individual programs and
reviews of other studies on the effectiveness of individual abstinence-
until-marriage education programs. In general, research studies on the
effectiveness of individual abstinence-until-marriage education
programs have examined the extent to which they changed participants'
demonstrated knowledge, declared intentions, and reported behavior
related to sexual activity and abstinence. For example, some studies
examined the impact of abstinence-until-marriage education programs on
participants' knowledge of concepts taught in the programs, as well as
participants' declared attitudes about abstinence and teen sex. Some
studies examined the impact of these programs on such outcomes as
participants' declared commitment to abstain from sex until marriage,
participants' understanding of the potential consequences of having
intercourse, and participants' reported ability to resist pressures to
engage in sexual activity. Some of the studies we reviewed examined the
impact of abstinence-until-marriage programs on participants' sexual
behavior, as measured, for example, by the proportion of participants
who reported having had sexual intercourse and the frequency of sexual
intercourse reported by participants. In general, the efforts to study
and build a body of research on the effectiveness of most abstinence
education programs have been under way for only a few years, in part
because grants under the two programs that account for the largest
portion of federal spending on abstinence education--the State Program
and the Community-Based Program--were not awarded until 1998 and 2001,
respectively.
Several Factors Limit the Conclusions That Can Be Drawn about the
Effectiveness of Abstinence-until-Marriage Education Programs:
Most of the efforts of HHS, states, and other researchers to evaluate
the effectiveness of abstinence-until-marriage education programs
included in our review have not met certain minimum criteria that
experts have concluded are necessary in order for assessments of
program effectiveness to be scientifically valid. For example, most of
the efforts included in our review did not include experimental or
quasi-experimental designs, nor did they measure behavioral or
biological outcomes. In addition, the results of some assessment
efforts that meet the criteria of a scientifically valid assessment
have varied, and two key studies that meet these criteria have not yet
been completed.
Experts Have Developed Criteria to Evaluate Efforts to Assess
Abstinence-until-Marriage Education Programs:
In an effort to better assess the merits of the studies that have been
conducted on the effectiveness of sexual health programs--including
abstinence-until-marriage education programs--scientific experts have
developed criteria that can be used to gauge the scientific rigor of
these evaluations. For example, in 2001, the National Campaign to
Prevent Teen Pregnancy--an organization focused on reducing teen
pregnancy--published a report by a panel of scientific experts that
assessed the evidence reported on abstinence-until-marriage education
programs in peer-reviewed journals and other literature.[Footnote 56]
The panel developed criteria that an evaluation of a program's
effectiveness must meet in order for the program's results to be
considered scientifically valid. In addition, in 2004, former U.S.
Surgeon General David Satcher convened a panel of experts to discuss,
among other things, best practices for evaluating the effectiveness of
sexual health education programs--including abstinence-until-marriage
education programs.[Footnote 57] This panel published a report in 2006
that describes similar scientific criteria that assessments of program
effectiveness need to meet in order for their results to be
scientifically valid. Further, experts we interviewed agreed that these
criteria are important for ensuring that the results of a study support
valid conclusions. In general, these panels, as well as the experts we
interviewed, agreed that scientifically valid studies of a program's
effectiveness should include the following characteristics:
* An experimental design that randomly assigns individuals or schools
to either an intervention group or control group, or a quasi-
experimental design that uses nonrandomly assigned but well-matched
comparison groups. According to the panel of scientific experts
convened by the National Campaign to Prevent Teen Pregnancy,
experimental designs or quasi-experimental designs with well-matched
comparison groups have at least three important strengths that are
typically not found in other studies, such as those that use aggregated
data: they evaluate specific programs with known characteristics, they
can clearly distinguish between participants who did and did not
receive an intervention, and they control for other factors that may
affect study outcomes. Therefore, experimental and quasi-experimental
study designs have a greater ability to assess the causal impact of
specific programs than other types of studies.[Footnote 58] According
to scientific experts, studies that include experimental or quasi-
experimental designs should also collect follow-up data for a minimum
number of months after subjects receive an intervention.[Footnote 59]
Experts reported that follow-up periods are important in order to
identify the effects of a program that are not immediately apparent or
to determine whether these effects diminish over time. In addition,
experts have reported that studies should have a sample size of at
least 100 individuals for study results to be considered scientifically
valid.[Footnote 60]
* Studies should assess or measure changes in biological outcomes or
reported behaviors instead of attitudes or intentions. According to
scientific experts, biological outcomes--such as pregnancy rates, birth
rates, and STD rates--and reported behaviors--such as reported
initiation and frequency of sexual activity--are better measures of the
effectiveness of abstinence-until-marriage programs, because adolescent
attitudes and intentions may or may not be indicative of actual
behavior. For example, adolescents may report that they intend to
abstain from sexual intercourse but may not actually do so.
Most Efforts to Assess the Effectiveness of Abstinence-until-Marriage
Education Programs Have Not Used an Experimental or Quasi-Experimental
Design:
Many of the efforts by HHS, states, and other researchers that we
identified in our review lack at least one of the characteristics of a
scientifically valid study of program effectiveness. That is, most of
the efforts to assess the effectiveness of these programs have not used
experimental or quasi-experimental designs with sufficient follow-up
periods and sample sizes to make their conclusions scientifically
valid. For example, ACF--and before it, HRSA--used, according to ACF
officials, grantee reporting on outcomes in order to monitor grantees'
performance, target training and technical assistance, and help
grantees improve service delivery. However, because the outcomes
reported by grantees have not been produced through experimentally or
quasi-experimentally designed studies, such information cannot be
causally attributed to any particular abstinence-until-marriage
education program. While ACF requires its fiscal year 2006 grantees of
the Community-Based Program to contract with third-party evaluators to
select and monitor outcomes for their programs, ACF is not specifically
requiring these grantees to use experimental or quasi-experimental
designs. Therefore, it is not clear whether these evaluations will
include such designs. Similarly, ACF's use of national data on
adolescent behavior and birth rates to assess its State and Community-
Based Programs is of limited value because these data do not
distinguish between those who participated in abstinence-until-
marriage education programs and those who did not. Consequently, these
national data sets, which represent state-reported vital statistics and
a nationwide survey of high school students, cannot be used to causally
link declines in birth rates and adolescent sexual activity to the
effects of specific abstinence-until-marriage education
programs.[Footnote 61]
Similarly, the efforts we identified by states and researchers to
assess the effectiveness of abstinence-until-marriage education
programs often did not include experimental or quasi-experimental
designs. None of the state evaluations we reviewed that have been
completed included randomly assigned control groups. For instance, one
state evaluation that we reviewed only included students who
volunteered to participate in the study. This evaluation report stated
that the absence of a randomly assigned control group in the evaluation
did not allow the evaluators to determine whether observed changes in
participants' reported sexual behavior--as indicated through surveys
administered at the beginning and end of a program--could be attributed
to the abstinence-until-marriage education program.[Footnote 62]
Similarly, some of the journal articles that we reviewed described
studies to assess the effectiveness of abstinence-until-marriage
programs that did not include experimental or quasi-experimental
designs needed to support scientifically valid conclusions about the
programs' effectiveness. In these studies, researchers administered
questionnaires to study participants before and after they completed an
abstinence-until-marriage education program and assessed the extent to
which the responses of participants changed.[Footnote 63] These studies
did not compare the responses of study participants with a group that
did not participate in an abstinence-until-marriage education program.
In addition, some of the studies used insufficient follow-up periods,
thereby limiting the conclusions that can be drawn about the
effectiveness of the abstinence-until-marriage education programs being
studied. For example, two journal articles that we reviewed described
studies that measured the effectiveness of abstinence-until- marriage
programs in delaying the initiation of sexual activity from 1 to 2
months after completion of the program.[Footnote 64] Scientific experts
consider this follow-up period too short to assess whether the programs
had a valid effect.
According to scientific experts, HHS, states, and other researchers
face a number of challenges in designing experimental or quasi-
experimental studies of program effectiveness. According to these
experts, experimental or quasi-experimental studies can be expensive
and time-consuming to carry out, and many grantees of abstinence-until-
marriage education programs have insufficient time and funding to
support these types of studies. Moreover, it can be difficult for
researchers assessing abstinence-until-marriage education programs to
convince school districts to participate in randomized intervention and
control groups, in part because of sensitivities to surveying
attitudes, intentions, and behaviors related to abstinence and sex. For
example, in a third-party evaluation of its program, one grantee of the
State Program originally planned to administer follow-up surveys 1 year
after participants finished their abstinence education program, but the
evaluators decided not to conduct this follow-up because of
confidentiality concerns and the difficulty of locating students. In
addition, the contractors hired to design ASPE's study of the
effectiveness of the Community-Based Program have reported difficulties
finding school districts that are willing to participate in randomly
assigned intervention and control groups receiving either abstinence-
until-marriage education or comprehensive sex education. An ASPE
official told us that although a "randomized approach" is the best
design for assessing the effectiveness of a program, the approach is
also the most difficult to conduct.
Most Efforts to Assess the Effectiveness of Abstinence-until-Marriage
Education Programs Have Not Measured Behavioral or Biological Outcomes:
Another factor that limits the conclusions that can be drawn about the
effectiveness of abstinence-until-marriage education programs is the
fact that most efforts in our review to study the effectiveness of
these programs did not measure changes in behavioral or biological
outcomes among participants. Instead, most of the efforts we identify
in our review used reported intentions and attitudes in order to assess
the effectiveness of abstinence-until-marriage programs. For example,
neither ACF's community-based grantees nor OPA's AFL grantees are
required to report on behavioral or biological outcomes, such as rates
of intercourse or pregnancy. Similarly, the journal articles we
reviewed were more likely to use reported attitudes and intentions--
such as study participants' reported attitudes about premarital sexual
activity or their reported intentions to remain abstinent until
marriage--rather than their reported behaviors or biological outcomes
to assess the effectiveness of abstinence-until-marriage programs. For
example, in one journal article we reviewed, participants were asked to
rate the likelihood that they would have sexual intercourse as
unmarried teenagers; another journal article described a study in which
participants rated the likelihood that they would have sexual
intercourse in the next year, before finishing high school, and before
marriage.[Footnote 65]
Experts, as well as state and HHS officials, have reported that it can
be difficult to obtain scientifically valid information on biological
outcomes and sexual behaviors. Specifically, experts have reported that
when measuring an abstinence-until-marriage education program's affect
on biological outcomes--such as reducing pregnancy or birth rates--it
is necessary to have large sample sizes in order to determine whether a
small change in biological outcomes is the result of the abstinence-
until-marriage education program. In addition, state and federal
officials told us that they have experienced difficulties obtaining
information on sexual behaviors because of the sensitive nature of the
information they were trying to collect. For example, one state
official told us that her state's effort to evaluate abstinence-until-
marriage education programs was only able to measure changes in
participants' reported attitudes, instead of behaviors, because the
evaluators needed to obtain consent from the parents of the program
participants in order to ask them about their sexual behavior. The
state official explained that the requirement to obtain consent from
parents raised issues of self-selection, and therefore state officials
decided to ultimately halt the study and only report on the attitudes
that they had measured. In another example, ACF's fiscal year 2006
budget justification reports that ACF has had some difficulty in
obtaining reliable data from state grantees, in part because questions
about teenage sexual behavior are sensitive. OPA officials also
acknowledged that many communities will not allow grantees to ask
program participants questions about their sexual behavior because the
communities believe such questions are too intrusive. One OPA official
said that such restrictions affect the agency's ability to measure
behavioral outcomes, explaining that OPA cannot measure what it cannot
ask about.
Results of Some Scientifically Valid Assessment Efforts Have Varied,
and Other Key Studies Have Not Been Completed:
Among the assessment efforts we identified are some studies that meet
the criteria of a scientifically valid effectiveness study. However,
results of these studies have varied, and this limits the conclusions
that can be drawn about the effectiveness of abstinence-until-marriage
education programs. Some researchers have reported that abstinence-
until-marriage education programs have resulted in adolescents
reporting having less frequent sexual intercourse or fewer sexual
partners.[Footnote 66] For example, in one study of middle school
students, participants in an abstinence-until-marriage education
program who had sexual intercourse during the follow-up period were 50
percent less likely to report having two or more sexual partners when
compared with their nonparticipant peers.[Footnote 67] In contrast,
other studies have reported that abstinence-until-marriage education
programs did not affect the reported frequency of sexual intercourse or
number of sexual partners.[Footnote 68] For example, one study of
middle school students found that participants of an abstinence-until-
marriage program were not less likely than nonparticipants at the 1
year follow-up to report less frequent sexual intercourse or fewer
sexual partners.[Footnote 69] In addition to these varied findings, one
study found that an abstinence-until-marriage program was effective in
delaying the initiation of sexual intercourse in the short term but not
long term.[Footnote 70] Experts with whom we spoke emphasized that
there are still too few scientifically valid studies completed to date
that can be used to determine conclusively which, if any, abstinence-
until-marriage programs are effective.
Additionally, among the assessment efforts we identified are some
studies that experts anticipate will meet the criteria of a
scientifically valid effectiveness study but are not yet completed. One
of these key studies is the final Mathematica report, contracted by
ASPE, on the State Program.[Footnote 71] The final report was
originally slated for publication in 2005, but an ASPE official stated
that the final report has been delayed until 2007 so that researchers
can extend the follow-up period to improve their response rate and the
reliability of the information they collect. Another key study is CDC's
research on middle school programs, which is not expected to be
completed until 2009.
Experts and federal officials we interviewed stated that they expect
the results of these two federally funded studies to add substantively
to the body of research on the effectiveness of abstinence-until-
marriage education programs. One expert with whom we spoke said that
she expects the final Mathematica report on participants' behaviors to
provide the groundwork for the field. Another expert we interviewed
stated that the CDC study was very well-designed and she expects the
results to contribute to the development of effective abstinence-until-
marriage education curricula.
Conclusions:
There have been various efforts--by HHS, states, and others--to assess
the scientific accuracy of educational materials used in abstinence-
until-marriage education programs and the effectiveness of these
programs. However, efforts to evaluate both the accuracy and
effectiveness of abstinence-until-marriage education programs have
been, in various ways, limited. ACF, which administers the two programs
that account for the largest portion of federal spending on abstinence-
until-marriage education, does not review or require its grantees to
review program materials for scientific accuracy. In addition, not all
grantees of the State Program have chosen to review their materials.
Because of these limitations, ACF cannot be assured that the materials
used in its State and Community-Based Programs are accurate. Moreover,
OPA, which reviews all grantees' proposed abstinence-until-marriage
educational materials, and states that review educational materials
have found inaccuracies in some educational materials used by
abstinence-until-marriage programs.
Similarly, most of the efforts described in our review to assess the
effectiveness of abstinence-until-marriage programs have not met
minimum scientific criteria needed to draw valid conclusions about
their effectiveness. Specifically, most efforts by agencies, states,
and other researchers have not included experimental or quasi-
experimental designs that can establish whether changes in behaviors or
biological outcomes can be causally linked to specific abstinence-
until-marriage education programs. While these types of studies are
time-consuming and expensive, experts said that they are the only
definitive way to draw valid conclusions about the effectiveness of
these programs. In addition, among the assessment efforts we identified
are some studies funded by HHS that experts anticipate will meet the
criteria of a scientifically valid effectiveness study but are not yet
completed. When completed, these HHS-funded studies may add
substantively to the body of research on the effectiveness of
abstinence-until-marriage education programs.
Recommendation for Executive Action:
To address concerns about the scientific accuracy of materials used in
abstinence-until-marriage education programs, we recommend that the
Secretary of HHS develop procedures to help assure the accuracy of such
materials used in the State and Community-Based Programs. To help
provide such assurances, the Secretary could consider alternatives such
as (1) extending the approach currently used by OPA to review the
scientific accuracy of the factual statements included in abstinence-
until-marriage education to materials used by grantees of ACF's
Community-Based Program and requiring grantees of ACF's State Program
to conduct such reviews or (2) requiring grantees of both programs to
sign written assurances in their grant applications that the materials
they propose using are accurate.
Agency Comments and Our Evaluation:
HHS provided written comments on a draft of this report. (See app.
III.)
In its written comments, HHS stated that it will consider requiring
grantees of both ACF programs to sign written assurances in grant
applications that the materials they use are accurate. Regarding
accuracy, HHS's written comments also noted that all applicants for
federal assistance attest on the application form--Standard Form 424--
that all data in their applications are "true and correct," and that in
the view of HHS, this applies to information presented in curricula
funded by federal grants. However, as we stated in the draft report,
grantees of the State Program are not required to submit curricula as a
part of their applications; therefore, the attestation in Standard Form
424 would not apply to curricula used by those grantees. In addition,
as stated in the draft report, some states have reviewed materials used
in abstinence-until-marriage education programs, but these reviews
occurred after they received funding from ACF. Further, while grantees
of the Community-Based Program were required to submit copies of their
curricula and a Standard Form 424 in fiscal year 2006 as part of their
applications, none of the materials specifically require an assurance
of scientific accuracy. Further, OPA and states have found inaccuracies
in educational materials used by abstinence-until-marriage programs.
HHS's written comments also stated that ACF requires that curricula
conform to HHS's standards grounded in scientific literature. HHS's
comments refer to the curriculum standards for this program that detail
what types of information must be included in abstinence-until-marriage
curricula, and the comments stated that the curricula must provide
supporting references for this information. Further, HHS's comments
stated that ACF staff review the curricula to ensure compliance with
these standards. The draft report stated this. However, a requirement
that curricula include certain types of information does not
necessarily ensure the accuracy of the scientific facts included in the
abstinence-until-marriage materials. For example, while education
materials may include information on failure rates associated with
contraceptives or STD infections, this information may be outdated or
otherwise inaccurate or incomplete. HHS's written comments also stated
that if it finds inaccurate statements during the review process or at
any time during the grant period, ACF works with grantees to take
corrective action. To ensure completeness, we have added this statement
to the report. Further, HHS stated that 2 inaccuracies cited in the
draft report had been corrected before our work began. We believe HHS
is referring to inaccuracies identified by OPA during its review of
materials for scientific accuracy and this reinforces the need for
review of materials used by ACF's grantees.
As HHS noted in its written comments, we did not define the term
scientific accuracy. HHS stated that it disagreed with certain findings
of the report because it was difficult to precisely determine the
criteria we employed in making the recommendation as to scientific
accuracy. As we stated in the scope and methodology section of the
draft report, the objective of our work was to focus on efforts by HHS
and states to review the accuracy of scientific facts included in
abstinence-until-marriage education materials. Performing an
independent assessment of the criteria used by these entities to
determine the scientific accuracy of education materials or the quality
of the reviews was beyond the scope of the work.
Regarding effectiveness, HHS's written comments also described a number
of actions it is taking to determine program effectiveness and improve
the quality of programs and research. Specifically, HHS's comments
described (1) studies undertaken or funded by ASPE, CDC, and NIH; (2)
technical assistance provided by OPA and ACF; (3) grantee evaluation
requirements; and (4) ACF and OPA requirements for the amount of grant
funds to be spent on evaluations. All of this information was included
in our draft report. HHS's comments also described a new effort funded
by ACF and ASPE that is designed to build capacity for quality research
in the field of abstinence education. We added information on this
effort to the report. HHS's written comments also describe evaluations
that resulted from an Abstinence Education Evaluation Conference
sponsored by ACF and OPA. While this conference was described in the
draft report, we added more detail regarding the content of the
conference. HHS's written comments also describe OPA's efforts to
assess the effectiveness of the AFL Program. We had included this
information in the draft report.
HHS's written comments stated that it may be too soon to draw
conclusions about the effectiveness of ACF's and OPA's programs, in
part, because key studies have not been completed. We agree and
discussed this in the draft report. As we noted in the draft report,
key studies funded by HHS that experts anticipate will meet the
criteria of a scientifically valid effectiveness study are not yet
completed, but when completed these HHS funded studies may add
substantively to the body of research on the effectiveness of
abstinence-until-marriage education programs.
In addition, the comments stated that having an inadequate amount of
scientifically valid and conclusive evaluation studies is not unique to
abstinence-until-marriage education programs, and a recent ASPE review
of comprehensive sex education programs found mixed results on their
effectiveness. However, the scope of our report was focused on
abstinence-until-marriage education programs, and we did not review
comprehensive sex education programs or make any comparisons between
the two types of programs.
HHS also provided technical comments, which we incorporated into the
report as appropriate.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after its date. We will then send copies of this report to the
Secretary of HHS and to other interested parties. In addition, this
report is available at no charge on GAO's Web site at [Hyperlink,
http://www.gao.gov]. We will also make copies available to others upon
request.
If you or your staff have any questions about this report, please call
me at (202) 512-3407 or crossem@gao.gov. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report. GAO staff who made major contributions to this
report are listed in appendix IV.
Signed by:
Marcia Crosse:
Director, Health Care:
List of Requesters:
The Honorable Henry A. Waxman:
Ranking Minority Member:
Committee on Government Reform:
House of Representatives:
The Honorable Pete Stark:
Ranking Minority Member:
Subcommittee on Health:
Committee on Ways and Means:
House of Representatives:
The Honorable Sherrod Brown:
Ranking Minority Member:
Subcommittee on Health:
Committee on Energy and Commerce:
House of Representatives:
The Honorable Barbara Boxer:
The Honorable Maria Cantwell:
The Honorable Richard J. Durbin:
The Honorable Russell D. Feingold:
The Honorable Dianne Feinstein:
The Honorable Tom Harkin:
The Honorable James M. Jeffords:
The Honorable Edward M. Kennedy:
The Honorable Frank R. Lautenberg:
The Honorable Patrick Leahy:
The Honorable Patty Murray:
United States Senate:
The Honorable Howard L. Berman:
The Honorable Lois Capps:
The Honorable Jay Inslee:
The Honorable Barbara Lee:
The Honorable Nita M. Lowey:
The Honorable Betty McCollum:
House of Representatives:
[End of section]
Appendix I: HRSA's Technical Assistance Contract for Abstinence
Education:
The Health Resources and Services Administration (HRSA) awarded a
contract to the National Abstinence Clearinghouse (NAC) in 2002 to
provide assistance with its Community-Based Abstinence Education
Program (Community-Based Program) and Abstinence Education Program
(State Program).[Footnote 72] NAC is a nonprofit educational
organization whose mission is to promote the appreciation for and
practice of sexual abstinence until marriage through the distribution
of age appropriate, factual, and medically referenced materials. The
purpose of the contract was (1) to develop national criteria for the
review of abstinence-until-marriage educational materials and to create
a directory of approved materials; (2) to provide medical accuracy
training to grantees; and (3) to provide technical support to grantees,
such as assistance with program evaluation.[Footnote 73]
We are reporting on the steps that HRSA took to award the contract to
NAC in response to concerns that have been raised by a congressional
requester. In general, these concerns centered on the extent to which
the selection process was competitive and whether HRSA identified the
potential for an organizational conflict of interest.
HRSA awarded the contract to address three concerns it had with the
Community-Based Program during 2001, the first year of its
implementation. First, HRSA officials needed guidance to determine
whether abstinence-until-marriage education materials conformed to the
definitional requirements of the Social Security Act.[Footnote 74]
Second, many grantees lacked the medical background and training to
ensure that they would provide medically accurate, science-based
information in their programs. Third, grantees also lacked experience
with the technical management of federal grants, including how to
conduct evaluations of their programs.
HRSA Used Full and Open Competition Procedures to Award the Contract to
NAC:
HRSA used full and open competition procedures to award the contract to
NAC. In doing so, HRSA (1) publicly solicited proposals from potential
contractors; (2) conducted technical evaluations of both the original
proposals and the revised proposals for those considered to be in the
competitive range; and (3) determined that NAC's proposal represented
the best overall value to the government. This process, which took
place from May 2002 through September 2002, resulted in HRSA awarding
NAC the contract with a potential value of nearly $2.7 million.
HRSA issued a notice on May 20, 2002, on the FedBizOpps Web site, the
government point of entry for notifying potential contractors of
federal contract opportunities, indicating its intent to publicly
request proposals from prospective contractors in June 2002.[Footnote
75] On June 20, 2002, HRSA posted the solicitation on the FedBizOpps
Web site indicating that the abstinence contract would be awarded using
full and open competition procedures, that is, all responsible
prospective contractors would be provided the opportunity to
compete.[Footnote 76] The solicitation, which was a Request for
Proposals (RFP), described the contract objectives, which included (1)
the development of national criteria for the review of abstinence-
until-marriage educational materials and the development of a directory
of approved materials; (2) the provision of medical accuracy training
to grantees; and (3) the provision of technical support to grantees,
such as assistance with program evaluation. The RFP stated that HRSA
intended to award a cost-reimbursement contract with fixed fee for a 1-
year base period and 2 option years.[Footnote 77] This was a best value
procurement; that is, HRSA reserved the right in the RFP to select for
award the proposal that HRSA determined offered the best value to the
government, even if it did not offer the lowest cost. Further, the RFP
stated that the technical evaluation of the prospective contractors'
proposals would receive paramount consideration in the selection of the
contractor. According to the RFP, this evaluation would include an
assessment of the prospective contractor's technical approach, the
organizational experience and expertise of the prospective contractor,
the plans for personnel and management of the work, and the prospective
contractor's statement and understanding of the project purpose. Other
factors, such as the estimated cost, past performance under other
contracts for similar services, and the subcontracting plan would also
be considered in the selection process. Five prospective contractors
submitted proposals to HRSA by July 31, 2002, when proposals were due.
HRSA established a review committee to conduct the technical evaluation
of the five proposals. This committee included three voting members and
a nonvoting chairperson. The Director of HRSA's Community-Based and
State Programs and two analysts from other programs within the
Department of Health and Human Services (HHS) served as the voting
members, and the chairperson of the review committee was a project
officer of HRSA's Community-Based Program. The committee members
conducted the technical evaluation of the proposals, according to the
criteria in the RFP, as described above. Three proposals with the
highest technical scores were determined to be in the competitive
range,[Footnote 78] with NAC's proposal receiving the highest technical
score. HRSA requested in writing that the competitive range offerors
address certain technical and cost issues and submit revised proposals
to HRSA by September 17, 2002.[Footnote 79] For example, HRSA requested
that one of the prospective contractors other than NAC clearly describe
its proposed management of day-to-day tasks of the contract and provide
justification for several labor and travel expenditures. HRSA did not
have oral discussions with the competitive range offerors. HRSA's
review committee evaluated the revised proposals and again gave NAC's
revised proposal the highest technical score.
Although NAC's estimated cost was not the lowest among the proposals in
the competitive range, HRSA determined that NAC had proposed a
realistic cost estimate for the contract. Accordingly, and in light of
the NAC proposal's high technical rating and the RFP's evaluation
criteria giving paramount consideration to the technical evaluation,
HRSA determined that NAC's proposal represented the best value to the
government. HRSA awarded a contract to NAC on September 27, 2002. The
contract had a 1-year base period of performance with an estimated
value of $854,681, and included 2 option years for a total potential
value of $2,673,784. According to a HRSA official, this cost-
reimbursement contract did not include a fee. All of the prospective
contractors were made aware that a debriefing to explain the selection
decision and contract award would be provided at their request. One
prospective contractor requested and received a debriefing from HRSA.
No protests were filed with the agency challenging the award of the
contract to NAC. There were no bid protests filed with GAO.[Footnote
80]
HRSA Identified No Actual or Potential Organizational Conflicts of
Interest:
HRSA officials told us that they did not identify any actual or
potential organizational conflicts of interest during the acquisition
process. As defined in the Federal Acquisition Regulation (FAR), an
organizational conflict of interest arises where:
* because of other activities or relationships, a person is unable or
potentially unable to provide impartial assistance or advice to the
government; or:
* the person's objectivity in performing the contract work is or might
be otherwise impaired; or:
* a person has an unfair competitive advantage.[Footnote 81]
An organizational conflict of interest may result when factors create
an actual or potential conflict of interest during performance of a
contract, or when the nature of the work to be performed under one
contract creates an actual or potential conflict of interest involving
a future acquisition.[Footnote 82] Under the FAR, contracting officers
are required to analyze planned acquisitions to identify and evaluate
potential organizational conflicts of interest as early in the
acquisition process as possible, and to take steps to avoid,
neutralize, or mitigate significant potential conflicts of interest
before a contract is awarded.[Footnote 83]
According to HRSA's contracting officer, HRSA did not identify any
actual or potential organizational conflicts of interest. In reaching
this conclusion, the contracting officer told us that he reviewed the
statement of work, including the background and objectives of the
proposed contract, the stated purpose of the contact, the criteria
established to evaluate the proposals, the past performance of the
competitors, and NAC's proposal. HRSA's contracting officer also told
us that he did not formally document his assessment of organizational
conflict of interest.[Footnote 84]
[End of section]
Appendix II: Methodology for Identifying and Reviewing Research
Studies:
To identify research studies that examine the effectiveness of
abstinence-until-marriage education programs among adolescents and
young adults, we searched two reference database systems, PubMed and
ProQuest. We used the following keywords to search for research studies
that were published from January 1, 1998, through May 22, 2006:
"virginity," "abstinence education," "abstinence and curriculum,"
"abstinence only," "teen pregnancy and prevention," and "abstinence
until marriage."[Footnote 85] We reviewed the research article titles
that were generated from the PubMed and ProQuest searches and
identified articles that appeared to focus on the evaluation of the
effectiveness of abstinence-until-marriage education programs. In cases
where we could not determine, based on the title, whether a study
appeared to focus on an abstinence-until-marriage education program
evaluation, we reviewed a summary of the article to obtain more
information about the research study. We also examined previous
summaries of the literature to identify additional research studies.
We then selected research studies for inclusion in our literature
review if they met three criteria. First, the study evaluated a group-
based, abstinence-until-marriage education program. We did not select
studies that evaluated one-on-one interactions, such as education
programs focused exclusively on parent-child interactions, or that
evaluated media campaigns. We reviewed the description of each
education program and curriculum, as described in the study, to
determine whether an abstinence-until-marriage education program was
being evaluated. Education programs that were described as including
detailed contraceptive information in their curricula, for example,
were not classified as abstinence-until-marriage programs. Second, the
study targeted adolescents and young adults in the United States, for
example, by indicating that participants in the evaluation were high
school or middle school students. Third, the study was a quantitative
rather than a qualitative evaluation of an abstinence-until-marriage
education program. We selected 13 research studies for inclusion in our
literature review.
We reviewed the selected research studies to obtain detailed
information about their methodologies and outcome variables. For
example, we determined whether each study used an experimental or quasi-
experimental design and whether the outcome measures included
attitudes, behavioral intentions, behaviors such as initiation of
sexual intercourse, or a combination of these.
[End of section]
Appendix III: Comments from the Department of Health and Human
Services:
Office Of The Assistant Secretary For Legislation:
Department Of Health & Human Services:
Washington, D.C. 20201:
SEP 22 2006:
Ms. Marcia Crosse:
Director, Health Care:
U.S. Government Accountability Office:
Washington, DC 20548:
Dear Ms. Crosse:
Enclosed are the Department's comments on the U.S. Government
Accountability Office's (GAO) draft report entitled, "Abstinence
Education: Efforts to Assets the Accuracy and Effectiveness of
Federally Funded Programs" (GAO-06-984), before its publication.
The Department provided several technical comments directly to your
staff.
These comments represent the tentative position of the Department and
are subject to reevaluation when the final version of this report is
received.
Sincerely,
Signed by:
Vincent J. Ventimiglia:
Assistant Secretary for Legislation:
Comments Of The Department Of Health And Human Services On The
Government Accountability Office's Draft Report Entitled "Abstinence
Education: Efforts To Assess The Accuracy And Effectiveness Of
Federally Funded Programs" (GAO-06-984):
GAO Recommendations:
GAO recommends that to address concerns about the scientific accuracy
of materials used in abstinence-until-marriage education programs, the
Secretary of HHS develop procedures to help assure the accuracy of such
materials used in the State and Community-based Programs. GAO further
recommends that to help provide such assurances, the Secretary could
consider alternatives such as (1) extending the approach currently used
by OPA to review the scientific accuracy of the factual statements
included in abstinence-until-marriage education materials used by
grantees of ACF's Community-based Programs and requiring grantees of
ACF's State Program to conduct such reviews, or (2) requiring grantees
of both programs to sign written assurances in their grant applications
that the materials they propose using are accurate.
HHS Comments:
HHS will consider requiring grantees of both ACF programs to sign
written assurances in grant applications that the materials they use
are accurate. However, as described 'below, ACF already requires
applicants to submit curricula that conform to standards grounded in
scientific data with their applications; and applicants for Federal
grant assistance already assure the government that all data in their
application, which includes data in curricula funded by a Federal
grant, are true and correct.
The report is divided into two main categories - scientific accuracy
and effectiveness of abstinence-until-marriage programs. Comments are
provided in that format.
Scientific Accuracy:
Although GAO was requested to "assess the scientific accuracy of
materials used in abstinence-until-marriage programs." (page 3), GAO
never defines the term "scientific accuracy" in the report. In fact,
the report specifically recognizes (page 3) that it does not attempt to
define or clarify the term. As such, it is difficult to precisely
determine the criteria employed by GAO in making the recommendations as
to scientific accuracy.
In part due to the lack of precision in the report's use of the term
"scientific accuracy," but also due to the report's incomplete
depiction of how such materials are developed and reviewed, HHS does
not agree with key claims of the report. One of the main claims of the
report, for example, involves the often-repeated claim "ACF. does not
review its grantees education materials for scientific accuracy, and
does not require grantees . to review their own materials for
scientific accuracy." (pages 5, 14, 39). HHS disagrees for the
following reasons:
First, all applicants for Federal assistance attest in their
applications that "all data in their applications are true and correct"
(Standard Form 424, [Hyperlink,
http://www.acf.hhs.gov/programs/ofs/forms.htm]). In the view of HHS,
this applies to information presented in curricula funded by Federal
grant funds.
Second, ACF requires its abstinence-education grantees to submit
educational curricula that conform to, and are thoroughly grounded in,
scientific literature. Indeed, the Fiscal Year 2006 Program
Announcement for the Community-Based Abstinence Education Program, the
largest of the three Federally funded abstinence programs, required
that successful applicants' curricula conform to very specific,
detailed curriculum standards (pages 5 through 13 of the program
announcement, [Hyperlink, http://www.acf.hhs.gov/grants/open/HHS-2006-
ACF-ACYF-AE-0099.html]). Each of these standards has robust backing in
scientific literature. Additionally, at least two of these standards
discussed specific requirements by which successful applicants'
curricula would adhere to scientific accuracy. Namely, the guidance
specifies that the curricula teach "the published failure rates
associated with contraceptives relative to pregnancy Prevention,
including ,real use' versus trial or `laboratory use,' human error,
product defect, teen use and possible side effects of contraceptives"
(Theme CI). Furthermore, the guidance for successful applicants'
curricula required that applicants' curricula teach "the epidemiology
of sexually transmitted diseases (STDs) in the U.S., e.g., infection
rates, modes of transmission, existence of incurable and potentially
fatal STDs" (Theme C2). Curricula must provide supporting references
for this information. Upon submission, ACF staff reviews the
application, including the curricula, to ensure compliance with these
scientifically valid standards. Should we find inaccurate statements
during the review process, or at any time during the grant project
period, ACF works with the grantees to take corrective action, as they
would do with any other ACF grantee.
Increased grantee awareness of the scientific and medical facts
relative to their delivery of abstinence education has been, and
continues to be, an important area of focus for ACF. For this reason,
ACF offers numerous training and technical assistance (T/TA)
opportunities to all of its grantees on a number of program-related
areas. In addition, as is the case with ACF's grantees in other program
areas, when ACF finds inaccuracies in a grantee's information or when
an inaccuracy is brought to the attention of ACF, whether during the
review process or at any time during the project period, ACF requires
and works with the grantee to have tie inaccuracy corrected as soon as
possible. In the few cases of which ACF is aware of inaccuracies found
to exist in abstinence materials, corrections were made once the
grantee became aware of those inaccuracies. For example, the two
inaccuracies cited in the GAO report were corrected before GAO even
began working on the report.
The GAO report presents an adequate assessment of OPA's review of
materials used in Adolescent Family Life (AFL) Abstinence Education
Programs. There are, However, a few points about this review process
that should be both clarified and highlighted. First, when OPA does
find inaccuracies in education materials proposed for use in an AFL
demonstration project, the grantee is notified and must make the
specified correction(s) before OPA approves their use. Similarly, OPA
grants include the condition, in each Notice of Grant Award issued for
AFL demonstration projects, that all educational materials must be
submitted to OPA for review and approval before they can be used in AFL
projects.
Aspects of the CDC review of materials for accuracy also warrant
additional clarification. Specifically, all materials developed and/or
used by national, State and local grantees are reviewed and approved by
the grantees' materials review panel. The review panel is a requirement
of the CDC cooperative agreement.
Effectiveness:
HHS is taking extensive action to determine program effectiveness and
improve the quality of programs and research. Work is being done
through the program offices, research agencies and through
Congressionally mandated studies. These efforts include the following:
* ASPE is overseeing a Congressionally mandated experimental evaluation
of abstinence education programs that will examine behavioral outcomes
over an extended period of time, and is in the process of designing
another large-scale longitudinal study that will also include an
experimental design and measure behavioral outcomes.
* A new effort that is not included in the report is the development if
a Center for Research and Evaluation on Abstinence Education. This is a
multi-year project funded by ACF and ASPE in FY 2006. It is designed to
build capacity for quality research in the field of abstinence
education. The main activities include a) conducting a needs assessment
to identify gaps in abstinence-education evaluation and technical
assistance needs; b) developing materials on abstinence-education
evaluation; c) delivering technical assistance and capacity- building
activities related to program evaluation; and d) developing several
research reports related to abstinence education. The goal is to
stimulate greater collaboration and quality implementation and outcome-
evaluation research of abstinence education programs.
* CDC is undertaking a study to evaluate the effectiveness of two
middle school curricula. The study will examine the relative
effectiveness of programs on behavioral outcomes.
* NIH is currently funding several studies related to the effectiveness
of abstinence education, which are underway.
* The two program offices (OPA and ACF) provide technical assistance
and fund research activities related to evaluation. In addition, two of
the grant programs (the AFL program and the Community-based Program)
currently have evaluation requirements of grantees - OPA allows up to
25 percent of grant funds awarded to be spent on evaluation and ACF is
now requiring that a minimum of 15 percent of funds awarded to be spent
on evaluation.
Although GAO does acknowledge that "ACF has made efforts to assess the
effectiveness of abstinence-until-marriage education programs." (page
20), GAO fails to mention two key elements of these efforts. First,
while GAO mentions the new evaluation requirement on Community- Based
Abstinence Education (CBAE) grantees, there is no mention in the report
that grantees are now required to spend at least 15 percent of their
Federal funding on evaluation activities. Depending on the amount of
the project, this amount could be up to $450,000 over five years for
each grant awarded. Never before have CBAE grantees been required to
comply with such a significant commitment to evaluation and research
activity. HHS believes that this requirement will dramatically improve
its grantees' ability to determine the effectiveness of the programs.
The improvements made to this year's guidance caused one peer-reviewed
scientific medical journal to claim that "...a rigorous evaluation of
project impact may now be possible."[Footnote 86] Second, the report
omits any mention of the abstinence-education evaluation; from the
first Abstinence Education Evaluation Conference sponsored by ACF and
OPA.[Footnote 87] The evaluations contained in this publication were
peer-reviewed for presentation at the conference and conform to many of
the standards published in the report (pages 31 through 33). ACF
recommends that GAO insert language in the report reflecting these
significant commitments to assessing program effectiveness.
While GAO also generally acknowledges that "OPA has also made efforts
assess the effectiveness of the AFL Program" it would be helpful to
specify what those efforts are since they, too, already conform to many
of the standards published in the report. [n particular, OPA has
developed performance measures and requires projects to develop outcome
objectives, conduct independent evaluations and use core evaluation
instruments.
Further, in response to the Office of Management and Budget's (OMB)
Program Assessment Review Tool (PART), OPA has developed performance
measures for their AFL program. Two of these performance measures apply
to AFL prevention demonstrations. AIL Prevention Core Evaluation
Instrument data, aggregated across prevention projects, will be used to
track them (Only the AFL projects, funded in fiscal year 2004 and
later, are required to use the core instruments, and thus, it will
require a few more funding cycles before tracking data are available
for all projects.)
Each AFL prevention-demonstration project is expected to develop
specific and measurable outcome objectives, two of which should address
the performance measures developed by OPA for prevention projects.
While these objectives are intended more as a way to focus the
individual project than as away to evaluate it, projects are expected
to tie their independent evaluations to their outcome objectives.
All AFL demonstration projects are required, by statute, to conduct an
independent evaluation of their project. The GAO report covers the many
difficulties these evaluation; face and OPA has taken steps, in the
past few years, to improve the quality of its AFL evaluations. The
limit of five percent of project funds for evaluation has been waived,
pursuant to statutory authority, and beginning in fiscal year 2004, new
projects can request up to 25 percent of project funds for evaluation.
Proposals considered for funding are held to commensurately more
rigorous criteria with respect to their evaluation plan. Core
evaluation instruments have been developed; all demonstration projects
are encouraged to use them and, beginning in fiscal year 2004, all new
demonstration projects funded by AFL were required to use them. The AFL
program also provides, through expert consultants, evaluation technical
assistance to all projects that require it.
OPA has developed two sets of core evaluation instruments for the AFL
program baseline and follow-up for both prevention and care
demonstrations to ensure that data, reflecting AFL legislative
requirements, are captured in the independent evaluations. The
instruments are intended as core only and additional instrumentation,
addressing specific and unique components of AFL demonstrations, can
also be used in the individual independent evaluations.
We note that because evaluation is a new requirement for ACF's
Abstinence Education Program and only recently intensified for OPA's
Abstinence Education Program, it may be too soon to be drawing
conclusions about their effectiveness. It would also be important to
note that because youth are typically 9-14 years of age in OPA's
Abstinence Education Program and 12-18 years of age in ACF's Abstinence
Education Program, following youth in both programs takes time. That
these large evaluations of the State and Community-based Programs are
still underway suggests that the conclusions drawn in the GAO report
may be premature or Speculative and may change significantly as new
information becomes available.
We agree with the report when it states that there are still too few
scientifically valid studies which have been completed, to date, and
which have been able to be used to determine conclusively which, if
any, abstinence-until-marriage programs have been effective We would
point out, however, that having an inadequate amount of scientifically
valid and conclusive evaluation studies as of yet is not unique to
abstinence until marriage education programs. A recent ASPE review of
comprehensive sex education programs show that while some have a
positive impact on some outcome measures, some have no impact and some
have a negative impact.
We believe that the answer to this challenge is not only a greater
emphasis on evaluation among grantees, but also the conducting of a
greater number of experimental programs generally. HHS would appreciate
greater mention of both of these efforts.
As HHS funds numerous abstinence demonstration projects across the
country, and as we do so with an emphasis on evaluation, we believe
that the field will mature to show the level of effectiveness and
impact of these programs. Furthermore, HHS eagerly awaits the results
of the first experimental trial referred to in the report, which will
undoubtedly strengthen service delivery within the abstinence-
education field, and benefit the youth who are choosing to remain
abstinent in ever-increasing numbers.
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
Marcia Crosse, (202) 512-7119 or crossem@gao.gov:
Acknowledgments:
In addition to the contact named above, Kristi Peterson, Assistant
Director; Kelly DeMots; Pam Dooley; Krister Friday; Julian Klazkin; and
Amy Shefrin made key contributions to this report.
FOOTNOTES
[1] Abstinence-until-marriage education programs also support HHS's
objective to promote family formation and healthy marriages.
[2] See K. L. Wilson et al., "A Review of 21 Curricula for Abstinence-
Only-Until-Marriage Programs," The Journal of School Health, vol. 75,
no. 3 (2005), and The Content of Federally Funded Abstinence-Only
Education Programs, United States House of Representatives, Committee
on Government Reform-Minority Staff, Special Investigations Division
(2004).
[3] See, for example, E. A. Borawski et al., "Effectiveness of
Abstinence-only Intervention in Middle School Teens," American Journal
of Health Behavior, vol. 29, no. 5 (2005), and J. B. Jemmott III, L. S.
Jemmott, and G. T. Fong, "Abstinence and Safer Sex HIV Risk-Reduction
Interventions for African American Adolescents: A Randomized Controlled
Trial," Journal of the American Medical Association, vol. 279, no. 19
(1998).
[4] The 10 states that received the largest share of funding in fiscal
year 2005 through the State Program were Arizona, Florida, Georgia,
Illinois, Louisiana, Michigan, New York, North Carolina, Ohio, and
Texas.
[5] See Douglas Kirby, Emerging Answers: Research Findings on Programs
to Reduce Teen Pregnancy (Washington, D.C.: National Campaign to
Prevent Teen Pregnancy, 2001). The experts identifying the design
characteristics of a scientifically valid study for the National
Campaign to Prevent Teen Pregnancy were drawn from institutions that
include the National Institutes of Health, the Medical Institute for
Sexual Health, the Alan Guttmacher Institute, the Institute for
Research and Evaluation, and various universities. See David Satcher,
The National Consensus Process on Sexual Health and Responsible Sexual
Behavior: Interim Report (Atlanta: Morehouse School of Medicine, 2006).
The panel convened by David Satcher included experts from a variety of
organizations, including the Medical Institute for Sexual Health, the
Alan Guttmacher Institute, and the American Academy of Pediatrics. In
addition, characteristics of a scientifically valid study have been
identified by other experts in the field of evaluation research. For
example, see Carol H. Weiss, Evaluation (Upper Saddle River: Prentice
Hall, 1998).
[6] HHS's Family Planning Program, for example, also supports the
objective to reduce the incidence of STDs and unintended pregnancies by
providing access to contraceptive supplies and family planning
information, especially for low-income persons, at community health
clinics. This program is authorized under Title X of the Public Health
Service Act.
[7] There are other federal sources of funding that are used for
abstinence education, such as the Temporary Assistance for Needy
Families (TANF) Program that is administered by ACF. Some states have
allocated some of their TANF funding for abstinence education programs.
For example, Florida has used TANF funds to provide community-based and
faith-based organizations with contracts to carry out abstinence
education. Other sources of funding that are used for abstinence
education include ACF's Compassion Capital Fund and CDC's Division of
Adolescent and School Health grants.
[8] 42 U.S.C. § 710(b)(2). This definition is also referred to as the A-
H definition. This statutory provision defines abstinence education for
purposes of the State Program. Annual appropriations acts and program
announcements have extended this definition to the Community- Based and
AFL Programs. See, e.g , Departments of Labor, Health and Human
Services, and Education, and Related Agencies Appropriations Act,
2006,Pub. L. No. 109-149,119 Stat. 2833, 2855-56.
[9] Through the State Program funds are also provided to insular areas
and the District of Columbia.
[10] Pub. L. No. 104-193, § 912; 110 Stat. 2353-54 (codified at 42
U.S.C. § 710).
[11] In this report, we refer to U.S. territories and commonwealths as
"insular areas."
[12] Some states and insular areas have not applied for funding under
the State Program. California, Maine, and Pennsylvania did not apply
for funding under the State Program in fiscal year 2005. In this
report, when we refer to "states," we are referring to all grantees of
the State Program--including states, insular areas, and the District of
Columbia.
[13] The Community-Based Program is conducted under section 1110 of the
Social Security Act. See 42 U.S.C. § 1310.
[14] In addition to the 63 grants awarded in fiscal year 2005, ACF is
also responsible for other grants that the agency awarded before 2005.
[15] See 42 U.S.C. § 300z et seq. The AFL Program also supports other
projects for pregnant and parenting adolescents, their infants, male
partners, and family members. The purpose of these projects is to
improve the outcomes of early childbearing for teen parents, their
infants, and their families. However, in this report, when we use the
term "AFL Program," we are referring only to the abstinence-until-
marriage component of the AFL Program.
[16] ACF awards formula grants under the State Program each year, and
states have 2 years to spend the funds they are awarded. In the
Community-Based Program and AFL Program, grantees develop multiyear
projects--up to 5 years--for which the first year of funding is
provided through competitive grants; for subsequent years, grantees may
obtain funding through noncompetitive continuation grants.
[17] In reviewing grantees applications, ACF does examine several
issues, including applicants' stated program goals and need for
assistance, their compliance with the A-H definition of abstinence
education, their intended approach in carrying out their objectives,
and their budget plan.
[18] HHS officials told us that if ACF finds inaccurate statements
during this more general review process or if inaccuracies are brought
to their attention at any time during the grant period, ACF officials
work with the grantees to take corrective action.
[19] In addition to reviewing materials for accuracy, one state
requires abstinence-until-marriage providers to sign a written
assurance that their materials are scientifically accurate. Officials
from this state also reported providing abstinence-until-marriage
education programs with public health consultants to provide technical
assistance and training to help ensure the accuracy of their
educational materials.
[20] In addition, some state officials we interviewed told us that
review committees for local school districts may review the scientific
accuracy of educational materials that include information about HIV
and other STDs, including abstinence-until-marriage education
materials.
[21] Mich. Comp. Laws Ann. § 380.1507b(2)(West 2004).
[22] See Ohio Department of Health, "Abstinence Education Program
Request for Proposals for Fiscal Year 2007," (program announcement,
2005).
[23] The human papillomavirus (HPV) causes an STD called genital HPV
infection.
[24] Some grantees of the Community-Based Program reported using more
than one of these curricula in fiscal year 2005.
[25] The administration of this contract was transferred to ACF in May
2005.
[26] The Medical Institute is a nonprofit organization that provides
educational resources, conferences, and seminars to educators, health
professionals, pregnancy care centers, and faith-based groups about
behaviors to decrease STDs and out-of-wedlock pregnancies, including
abstinence.
[27] See generally Bowen v. Kendrick, 487 U.S. 589 (1988), 657 F. Supp
1547 (D.D.C. 1987).
[28] In addition, a CDC official told us that some of its grantees are
producing educational materials with CDC funds to be used by abstinence-
until-marriage education programs, which are likely to include State
and Community-Based Program grantees. These materials are required to
be reviewed for scientific accuracy. CDC officials told us that they
have made corrections to some of these materials. Materials used in
school-based HIV prevention education programs that are supported with
CDC funds are also reviewed for scientific accuracy. A CDC official
told us that some of these programs are abstinence-until- marriage
education programs.
[29] In addition to OPA and some states, others have also reviewed the
scientific accuracy of abstinence-until-marriage education materials.
See, for example, Wilson et al.
[30] This reporting is a part of ACF's efforts to collect evaluative
information about these programs. These efforts include both
performance measurement--the ongoing monitoring and reporting of
program accomplishments toward preestablished goals--and program
evaluation--individual systematic studies to assess how well a program
is working. Both types of assessments aim to support decisions to
improve service delivery and program effectiveness. See GAO,
Performance Measurement and Evaluation: Definitions and Relationships,
GAO-05-739SP (Washington, D.C.: May 2005), for more information on
types of assessments.
[31] Previously, to report on the four measures, states have relied on
either state or national data sources, such as CDC's Youth Risk
Behavior Surveillance System.
[32] For example, in fiscal year 2002, state grantees developed such
measures as the percentage of teens surveyed who show an increase in
participating in structured activities after school hours; the
percentage of live births to women younger than 18, fathered by men age
20 and older; the percentage of program participants proficient in
refusal skills; the percentage of high school students who reported
using drugs or alcohol before intercourse; and the percentage of high
school students who had sexual intercourse for the first time before
age 13.
[33] The fiscal year 2006 program announcement for the Community-Based
Program provides examples of outcome measures that grantees could use,
including increased knowledge of the benefits of abstinence, the number
of youths who commit to abstaining from premarital sexual activity, and
increased knowledge of how to avoid high-risk situations and risk
behaviors.
[34] Fiscal year 2006 Community-Based Program grantees are required to
devote a minimum of 15 percent of their requested budgets to
performance monitoring by third-party contractors.
[35] Some grantees of the Community-Based Program may have to meet
reporting requirements established by HRSA. Grants under this program
are awarded for projects that may extend over a period of several
years. Grantees that were awarded grants when HRSA administered the
program and have since received noncompetitive continuation grants for
these projects are required to meet the reporting requirements in place
at the time they first received the competitively awarded grants.
[36] In fiscal year 2001, HRSA required grantees of the Community-Based
Program to report on the following four performance measures: the
proportion of program participants who successfully complete or remain
enrolled in an abstinence-only education program; the proportion of
program participants who have engaged in sexual intercourse; the
proportion of program participants who report a reduction in risk
behaviors, such as tobacco, alcohol, and drug use; and the rate of
births to female program participants.
[37] Specifically, HRSA required grantees of the Community-Based
Program to report annually on the following six performance measures:
the proportion of program participants who successfully completed or
remained enrolled in an abstinence-only education program; the
proportion of adolescents who understood that abstinence from sexual
activity is the only certain way to avoid out-of-wedlock pregnancy and
STDs; the proportion of adolescents who indicated an understanding of
the social, psychological, and health gains to be realized by
abstaining from premarital sexual activity; the proportion of
participants who reported that they had the skills necessary to resist
sexual urges and advances; the proportion of youth who committed to
abstaining from sexual activity until marriage; and the proportion of
participants who intended to avoid situations and risk behaviors, such
as drug use and alcohol consumption, which make them more vulnerable to
sexual advances and urges.
[38] In addition to these outcomes, grantees of the AFL Program are
required to report on program outputs, such as the number of program
participants, the average number of participants per session, and the
average number of sessions attended by participants. Agency officials
stated that OPA has implemented a new format for its grantees' reports,
which is intended to standardize their reporting on these outputs.
[39] OPA's grantees are required to perform evaluations of their
programs that are directly tied to their program objectives. For these
evaluations, OPA requires grantees to develop research hypotheses that
reflect the outcomes the grantees intend to achieve. This type of
evaluation is generally considered to be an outcome evaluation--which
assesses the extent to which a program achieves its outcome-oriented
objectives. These evaluations focus on outputs and outcomes to judge
program effectiveness but may also assess program process to understand
how outcomes are produced. In addition, grantees of the AFL Program are
required to perform implementation evaluations.
[40] OPA officials stated that grantees may also supplement the
standardized questionnaire with additional data collection instruments
to obtain information on the effectiveness of their abstinence-until-
marriage education programs.
[41] OPA has required that its grantees perform independent evaluations
of their programs since the program first awarded grants in 1982, and
requires that grantees devote from 1 percent to 5 percent of grant
funds to the evaluation of their programs. In cases where a more
rigorous or comprehensive evaluation is proposed, OPA may allow these
grantees to use up to 25 percent of their grant funds.
[42] A control group is a group of individuals or communities in a
study that is compared to an intervention group--a group in a study
that is receiving or participating in the program being studied. A
control group is a randomly assigned group that does not receive the
program. A comparison group is not randomly assigned like a control
group. However, individuals or communities in well-matched comparison
groups should have similar characteristics.
[43] Specifically, OPA requires that third-party evaluations of
grantees of the AFL Program compare, when possible, randomized control
or matched comparison groups with groups receiving abstinence-until-
marriage education. In addition, OPA requires that these evaluations
include a sufficient sample size to ensure that any observed
differences between the groups are statistically valid and that the
evaluations include a follow-up assessment of program participants at
least 6 months after the abstinence-until-marriage intervention has
been tested.
[44] OPA prepares annual performance reports as a part of HHS's
responsibilities under the Government Performance and Results Act,
which include program performance measures to help link funding
decisions with performance and review of related outcome measures.
[45] Data on teen birth rates and adolescents' reported sexual behavior
are contained in the National Vital Statistics System and the Youth
Risk Behavior Surveillance System, respectively. The former is a
national data set of public health statistics reported by states to
CDC, and the latter is a national data set based on nationwide surveys
administered to high school students by CDC.
[46] In order to estimate pregnancy rates among adolescents, states use
data on both birth rates and abortions among adolescents.
[47] ACF does require grantees of the State Program to describe, in
their grant applications, their plans for "tracking activities and
measuring achievement" of their program goals and objectives.
[48] See, for example, LeCroy & Milligan Associates, Inc., Abstinence
Only Education Program: Fifth Year Evaluation Report, a report prepared
for the Arizona Department of Health Services, 2003; Patricia Goodson
et al., Abstinence Education Evaluation: Phase 6, a report prepared for
the Texas Department of State Health Services, 2005; MGT of America,
Evaluation of Georgia Abstinence Education Programs Funded Under Title
V, Section 510, a report prepared for the Georgia Department of Human
Resources, 2005; Thomas E. Smith, It's Great to Wait: An Interim
Evaluation, a report prepared for the Florida Department of Health,
2001.
[49] According to the House Appropriations Committee report
accompanying the fiscal year 2005 appropriation act for Labor, HHS,
Education, and related agencies, the effectiveness of abstinence
education programs should be determined by measures that include the
prevention and reduction of adolescent pregnancies and STD infections,
age at first sexual activity and intercourse, frequency of sexual
activity and intercourse, and numbers who postpone sexual activity or
intercourse through adolescence. See H.R. Rep. No. 108-636, at 139-140
(2004).
[50] According to ASPE officials, one factor that has contributed to
delays in the initiation of this study is the difficulty in recruiting
schools to participate.
[51] The five abstinence-until-marriage education programs being
studied are My Choice, My Future! in Powhatan, Virginia; ReCapturing
the Vision in Miami, Florida; Teens in Control in Clarksdale,
Mississippi; Families United to Prevent Teen Pregnancy in Milwaukee,
Wisconsin; and Heritage Keepers in Edgefield, South Carolina.
[52] See B. Devaney et al., The Evaluation of Abstinence Education
Programs Funded Under Title V Section 510: Interim Report, a report
prepared for ASPE, 2002.
[53] See R. Maynard et al., First-Year Impacts of Four Title V, Section
510 Abstinence Education Programs, a report prepared for ASPE, 2005.
Mathematica's report on the first-year impacts of selected state
abstinence-until-marriage education programs focused on intermediate
outcomes, including attitudes about abstinence, teen sex, and marriage;
perceived consequences of teen and nonmarital sex; and expectations to
abstain from sexual intercourse.
[54] An impact evaluation assesses the net effect of a program by
comparing program outcomes with an estimate of what would have happened
in the absence of the program. This form of evaluation is employed when
external factors are known to influence the program's outcomes, in
order to isolate the program's contribution to achievement of its
objectives.
[55] HHS officials told us that the two curricula being tested are
intended to be comparable in length, intensity, and other
characteristics.
[56] See Kirby. This panel included experts from NIH, the Medical
Institute for Sexual Health, the Alan Guttmacher Institute, the
Institute for Research and Evaluation, and various universities.
[57] See Satcher. This panel included experts from a variety of
organizations, including the Medical Institute for Sexual Health, the
Alan Guttmacher Institute, and the American Academy of Pediatrics.
[58] For example, experts have reported that the use of randomly
assigned intervention and control groups is particularly important when
assessing the effectiveness of abstinence-until-marriage programs
because adolescents who voluntarily participate in such programs may be
self-selecting--that is, they may be more willing to accept the
principles of--and be influenced by--such programs when compared with
other adolescents. Mathematica's interim report on the evaluation of
the State Program noted that selection bias can "seriously undermine
the credibility" of study results.
[59] For example, one expert reported that studies assessing program
effectiveness should obtain information on participants for at least 3
months after the conclusion of a program when they are measuring
behaviors that can change quickly, such as frequency of sex. For
behaviors or outcomes that change less quickly, such as initiation of
sex or pregnancy rates, information on participants should be collected
for at least 6 months after the conclusion of a program.
[60] The panel of experts convened by the National Campaign to Prevent
Teen Pregnancy agreed that large sample sizes are necessary to
determine the magnitude of any discernable program effect and to ensure
that results of any study of effectiveness are statistically valid.
[61] In addition, according to ACF and CDC officials, it is difficult
to draw conclusions from national data sets about the effectiveness of
abstinence-until-marriage education programs because the national
survey questions used to produce these data often do not identify the
specific type of program or intervention survey respondents may have
participated in or received.
[62] See Goodson et al.
[63] See, for example, S. M. Fitzgerald et al., "Effectiveness of the
Responsible Social Values Program for 6th Grade Students in One Rural
School District," Psychological Reports, vol. 91 (2002), and J. E.
Barnett and C. S. Hurst, "Abstinence Education for Rural Youth: An
Evaluation of the Life's Walk Program, " The Journal of School Health,
vol. 73, no. 7 (2003).
[64] See, for example, D. A. Zanis, "Use of a Sexual Abstinence Only
Curriculum with Sexually Active Youths," Children & Schools, vol. 27,
no. 1 (2005), and G. Denny et al., "An Evaluation of An Abstinence
Education Curriculum Series: Sex Can Wait," American Journal of Health
Behavior, vol. 26, no. 5 (2002).
[65] See, for example, L. Sather and K. Zinn, "Effects of Abstinence-
Only Education on Adolescent Attitudes and Values Concerning Premarital
Sexual Intercourse, " Family and Community Health, vol. 25, no. 2
(2002), and G. Denny, M. Young, and C. E. Spear, "An Evaluation of the
Sex Can Wait Abstinence Education Curriculum Series," American Journal
of Health Behavior, vol. 23, no. 2 (1999).
[66] See Borawski et al. See also T. L. St. Pierre et al., "A 27-Month
Evaluation of a Sexual Activity Prevention Program in Boys & Girls
Clubs Across the Nation," Family Relations, vol. 44, no. 1 (1995).
[67] See Borawski et al.
[68] See N.G. Harrington et al., "Evaluation of the All Stars Character
Education and Problem Behavior Prevention Program: Effects on Mediator
and Outcome Variables for Middle School Students," Health Education &
Behavior, vol. 28, no. 5 (2001). See also Jemmott, Jemmott, and Fong.
[69] See Harrington et al.
[70] See Jemmott, Jemmott, and Fong and J. B. Jemmott III, L. S.
Jemmott, and G. T. Fong, reply to letter to editor, Journal of the
American Medical Association, vol. 281, no. 16 (1999), 1487. This study
found that an abstinence-until-marriage program delayed the initiation
of sexual intercourse at the 3-month follow-up period but not at 6 and
12 months.
[71] According to several scientific experts, Mathematica's study is an
important one, in part because of its sound design: the study randomly
assigns and compares control groups with groups receiving abstinence-
until-marriage education and uses surveys to follow up with program
participants for several months after their completion of a program.
[72] The administration of this contract was transferred from HRSA to
the Administration of Children and Families (ACF) in May 2005.
[73] The contract resulted in the development of criteria for reviewing
abstinence-until-marriage educational materials, and ACF included these
criteria in the fiscal year 2006 program announcement for the Community-
Based Program. According to the announcement, ACF will evaluate grant
applicants' proposed educational materials to ensure compliance with
the criteria. Medical accuracy training and technical support were
provided to grantees as a result of the contract. According to an ACF
official, a directory of approved abstinence-until- marriage
educational materials was not completed.
[74] Section 510(b)(2) of the Social Security Act defines abstinence
education as an educational or motivational program that: A. has as its
exclusive purpose, teaching the social, psychological, and health gains
to be realized by abstaining from sexual activity; B. teaches
abstinence from sexual activity outside marriage as the expected
standard for all school age children; C. teaches that abstinence from
sexual activity is the only certain way to avoid out-of-wedlock
pregnancy, sexually transmitted diseases, and other associated health
problems; D. teaches that a mutually faithful monogamous relationship
in context of marriage is the expected standard of human sexual
activity; E. teaches that sexual activity outside of the context of
marriage is likely to have harmful psychological and physical effects;
F. teaches that bearing children out-of-wedlock is likely to have
harmful consequences for the child, the child's parents, and society;
G. teaches young people how to reject sexual advances and how alcohol
and drug use increases vulnerability to sexual advances; and H. teaches
the importance of attaining self-sufficiency before engaging in sexual
activity.
[75] The current FedBizOpps Web site address is [Hyperlink,
http://www.fbo.gov/. Prior to October 1, 2005, the Web site address was
http://www.eps.gov/. The solicitation number was 240-MCHB-012(02)-abg.
[76] The Federal Acquisition Regulation (FAR) requires the contracting
officer to ensure that prospective contractors are responsible. FAR §
9.103. A responsible source refers to a prospective contractor that
has, among other things, adequate financial resources, the necessary
experience and technical skills to perform the work of the contract, a
satisfactory performance record, and the ability to meet the delivery
schedule. FAR § 9.104-1.
[77] Cost-reimbursement contracts are used only when uncertainties
involved in contract performance do not permit costs to be estimated
with sufficient accuracy to use any type of fixed-price contract. FAR §
16.301-2.
[78] Based on the ratings of each proposal against all evaluation
criteria, the contracting officer establishes a competitive range
consisting of all of the most highly rated proposals. FAR §
15.306(c)(1).
[79] HRSA officials, including an auditor, reviewed the cost proposals
in the competitive range.
[80] GAO's Office of General Counsel resolves disputes concerning
awards of federal contracts, which are known as bid protests.
[81] FAR § 2.101.
[82] FAR § 9.502(c).
[83] The FAR requires contracting officers to exercise common sense,
good judgment, and sound discretion in determining whether a
significant potential conflict of interest exists. FAR § 9.505.
[84] The FAR requires HRSA's contracting officers to formally document
their assessment only when a substantive issue concerning a potential
organizational conflict of interest exists. FAR § 9.504(d). HHS
acquisition regulations do not explicitly address the assessment of
organizational conflict of interest. Therefore, FAR subpart 9.5 is the
controlling regulation when HHS encounters an issue related to an
organizational conflict of interest.
[85] ACF first awarded grants under the State Program in fiscal year
1998.
[86] Michael Young & Tina Penhollow, "The Impact of Abstinence
Education: What Does the Research Say?" Journal of Health Education 37,
no. 4 (July 1, 2006).
[87] Evaluating Abstinence Education Programs: Improving Implementation
and Assessing In pact. Compiled manuscripts from "Strengthening
Abstinence Education Programs through Scientific Evaluation," November
3-4, 2005.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
