Public Transportation
Improvements Are Needed to More Fully Assess Predicted Impacts of New Starts Projects Gao ID: GAO-08-844 July 25, 2008Through the New Starts program, the Federal Transit Administration (FTA) evaluates and recommends new fixed guideway transit projects for funding using the evaluation criteria identified in law. In August 2007, FTA issued a Notice of Proposed Rulemaking (NPRM), in part, to incorporate certain provisions within the Safe, Accountable, Flexible, Efficient, Transportation Equity Act: A Legacy for Users (SAFETEA-LU) into the evaluation process. SAFETEA-LU requires GAO to annually review FTA's New Starts process. This report discusses (1) the information captured by New Starts project justification criteria, (2) challenges FTA faces as it works to improve the New Starts program, and (3) options for evaluating New Starts projects. To address these objectives, GAO reviewed statutes, FTA guidance and regulations governing the New Starts program, and interviewed experts, project sponsors, and Department of Transportation (DOT) officials.
FTA primarily uses cost-effectiveness and land use criteria to evaluate New Starts projects, but concerns have been raised about the extent to which the measures for these criteria capture total project benefits. FTA's current transportation system user benefits measure, which assesses a project's cost effectiveness, focuses on how proposed projects will improve mobility by reducing the real and perceived cost of travel. FTA told GAO that such mobility improvements are a critical goal of all transit projects. While the literature and most experts that GAO consulted with generally agree with this assertion, they also raised concerns that certain benefits are not captured. As a result, FTA may be underestimating transit projects' total benefits, but it is unclear the extent to which this impacts FTA's evaluation and rating process. FTA officials acknowledged many of these limitations but noted that resolving these issues would be difficult without a substantial investment of resources by all levels of government to improve and update local travel models. FTA faces several systemic challenges to improving the New Starts program, including addressing multiple program goals, limitations in local travel models, the need to maintain the rigor while minimizing the complexity of the evaluation process, and developing clear and consistent guidance for incorporating qualitative information. The evaluation criteria identified in the law reflect multiple goals for the program, which has led to varying expectations between FTA and project sponsors about what types of projects should be funded. Also, models that generate local travel demand forecasts are limited and may not provide all of the information needed to properly evaluate transit projects. FTA has taken steps to mitigate the modeling limitations, such as incorporating proxy measures to account for certain project impacts and developing a request for proposals to improve local travel models so that they can better predict changes in highway user benefits. However, according to FTA officials, the request for proposals is only a first step in improving local travel models, and additional resources are needed. Experts and project sponsors GAO interviewed discussed different options for evaluating proposed transit projects but identified significant limitations of each option. One option is to revise the current New Starts evaluation process as proposed by FTA in the August 2007 NPRM. While some experts GAO spoke to appreciated the rigor of the current evaluation process, others noted that the NPRM may still underestimate total project benefits. For example, FTA's measure of mobility improvements does not account for benefits accruing to highway users, and its measures of environmental benefits may not properly distinguish among projects. Experts also discussed other options for evaluating proposed transit projects, including benefit-cost analysis. Unlike FTA's current evaluation process, benefit-cost analysis would attempt to monetize all benefits and costs, which experts told GAO would be a more comprehensive approach to evaluating projects. FTA is currently prohibited by statute from considering the dollar value of mobility improvements in evaluating projects.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-08-844, Public Transportation: Improvements Are Needed to More Fully Assess Predicted Impacts of New Starts Projects
This is the accessible text file for GAO report number GAO-08-844
entitled 'Public Transportation: Improvements Are Needed to More Fully
Assess Predicted Impacts of New Starts Projects' which was released on
July 25, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
July 2008:
Public Transportation:
Improvements Are Needed to More Fully Assess Predicted Impacts of New
Starts Projects:
GAO-08-844:
GAO Highlights:
Highlights of GAO-08-844, a report to congressional committees.
Why GAO Did This Study:
Through the New Starts program, the Federal Transit Administration
(FTA) evaluates and recommends new fixed guideway transit projects for
funding using the evaluation criteria identified in law. In August
2007, FTA issued a Notice of Proposed Rulemaking (NPRM), in part, to
incorporate certain provisions within the Safe, Accountable, Flexible,
Efficient, Transportation Equity Act: A Legacy for Users (SAFETEA-LU)
into the evaluation process. SAFETEA-LU requires GAO to annually review
FTA‘s New Starts process. This report discusses (1) the information
captured by New Starts project justification criteria, (2) challenges
FTA faces as it works to improve the New Starts program, and (3)
options for evaluating New Starts projects. To address these
objectives, GAO reviewed statutes, FTA guidance and regulations
governing the New Starts program, and interviewed experts, project
sponsors, and Department of Transportation (DOT) officials.
What GAO Found:
FTA primarily uses cost-effectiveness and land use criteria to evaluate
New Starts projects, but concerns have been raised about the extent to
which the measures for these criteria capture total project benefits.
FTA‘s current transportation system user benefits measure, which
assesses a project‘s cost effectiveness, focuses on how proposed
projects will improve mobility by reducing the real and perceived cost
of travel. FTA told GAO that such mobility improvements are a critical
goal of all transit projects. While the literature and most experts
that GAO consulted with generally agree with this assertion, they also
raised concerns that certain benefits are not captured. As a result,
FTA may be underestimating transit projects‘ total benefits, but it is
unclear the extent to which this impacts FTA‘s evaluation and rating
process. FTA officials acknowledged many of these limitations but noted
that resolving these issues would be difficult without a substantial
investment of resources by all levels of government to improve and
update local travel models.
FTA faces several systemic challenges to improving the New Starts
program, including addressing multiple program goals, limitations in
local travel models, the need to maintain the rigor while minimizing
the complexity of the evaluation process, and developing clear and
consistent guidance for incorporating qualitative information. The
evaluation criteria identified in the law reflect multiple goals for
the program, which has led to varying expectations between FTA and
project sponsors about what types of projects should be funded. Also,
models that generate local travel demand forecasts are limited and may
not provide all of the information needed to properly evaluate transit
projects. FTA has taken steps to mitigate the modeling limitations,
such as incorporating proxy measures to account for certain project
impacts and developing a request for proposals to improve local travel
models so that they can better predict changes in highway user
benefits. However, according to FTA officials, the request for
proposals is only a first step in improving local travel models, and
additional resources are needed.
Experts and project sponsors GAO interviewed discussed different
options for evaluating proposed transit projects but identified
significant limitations of each option. One option is to revise the
current New Starts evaluation process as proposed by FTA in the August
2007 NPRM. While some experts GAO spoke to appreciated the rigor of the
current evaluation process, others noted that the NPRM may still
underestimate total project benefits. For example, FTA‘s measure of
mobility improvements does not account for benefits accruing to highway
users, and its measures of environmental benefits may not properly
distinguish among projects. Experts also discussed other options for
evaluating proposed transit projects, including benefit-cost analysis.
Unlike FTA‘s current evaluation process, benefit-cost analysis would
attempt to monetize all benefits and costs, which experts told GAO
would be a more comprehensive approach to evaluating projects. FTA is
currently prohibited by statute from considering the dollar value of
mobility improvements in evaluating projects.
What GAO Recommends:
GAO recommends that the Secretary of Transportation take steps to
improve the New Starts evaluation process, including seeking additional
resources to improve local travel models and seeking a legislative
change to allow FTA to consider the dollar value of mobility
improvements in evaluating projects.
DOT officials generally agreed with the findings and recommendations in
this report.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-844]. For more
information, contact Katherine Siggerud at (202) 512-2834 or
siggeurdk@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
FTA's Project Evaluation Measures Include a Range of Information, but
Not All Project Benefits Are Fully Captured:
FTA Faces Several Systemic Challenges to Improving the New Starts
Program:
Different Options for Evaluating Proposed New Starts Projects Exist,
but All Have Limitations:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Summary of New Starts and Small Starts Projects Evaluated,
Rated, and Recommended for Funding for FY 2009:
Appendix II: Scope and Methodology:
Appendix III: Explanation of FTA's Calculation of Transportation System
User Benefits:
Appendix IV: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: FTA's Current Project Justification Criteria and Measures for
Evaluating and Rating New Starts Projects:
Table 2: FTA's Proposed Project Justification Measures for Evaluating
and Rating New Starts Projects:
Table 3: Extent to Which FTA's Proposed Evaluation Measures Address
NPRM Stakeholder Concerns:
Table 4: Pending FFGAs and Projects in Final Design and Preliminary
Engineering:
Table 5: Fiscal Year 2009 Small Starts and Very Small Starts Funding
Recommendations:
Table 6: New Starts and Small Starts Project Sponsors Interviewed:
Table 7: Experts Interviewed for Fiscal Year 2009 New Starts Review:
Figures:
Figure 1: FTA's Current New Starts Evaluation Process:
Figure 2: Timeline of FTA's Implementation of SAFETEA-LU Changes:
Figure 3: Example of a TSUB Calculation:
Figure 4: FTA's Proposed New Starts Evaluation Process:
Figure 5: Examples of Potential Goals for Transit Projects:
Figure 6: Allocation of Administration's Proposed Fiscal Year 2009
Budget for New Starts:
Abbreviations:
ANPRM: Advanced Notice of Proposed Rulemaking:
BRT: Bus Rapid Transit:
DOT: Department of Transportation:
EPA: Environmental Protection Agency:
FFGA: full funding grant agreement:
FHWA: Federal Highway Administration:
FTA: Federal Transit Administration:
LRT: Light Rail Transit:
MPO: metropolitan planning organization:
NPRM: Notice of Proposed Rulemaking:
SAFETEA-LU: Safe, Accountable, Flexible, Efficient Transportation
Equity Act: A Legacy for Users:
TRB: Transportation Research Board:
TSUB: transportation system user benefits:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
July 25, 2008:
The Honorable Christopher J. Dodd:
Chairman:
The Honorable Richard C. Shelby:
Ranking Member:
Committee on Banking, Housing, and Urban Affairs:
United States Senate:
The Honorable James L. Oberstar:
Chairman:
The Honorable John L. Mica:
Ranking Republican Member:
Committee on Transportation and Infrastructure:
House of Representatives:
Since the early 1970s, a significant portion of the federal
government's share of new capital investment in mass transportation has
come through the Federal Transit Administration's (FTA) New Starts
program. Through this program, FTA identifies and recommends new fixed-
guideway transit projects for grants, typically through full funding
grant agreements (FFGA).[Footnote 1] Over the last decade, the New
Starts program has provided state and local agencies with over $10
billion to help design and construct transit projects throughout the
country.
The Safe, Accountable, Flexible, Efficient Transportation Equity Act: A
Legacy for Users (SAFETEA-LU) authorized the New Starts program through
fiscal year 2009. Although SAFETEA-LU maintained a number of program
requirements imposed by previous authorizing legislation, it also made
some changes to the program.[Footnote 2] For example, FTA must continue
to prioritize projects for funding by evaluating, rating, and
recommending potential projects on the basis of specific local
financial commitment and project justification criteria, including cost-
effectiveness, operating efficiencies, land use, mobility improvements,
and environmental benefits. SAFETEA-LU, however, also added economic
development as a project justification criterion.
We have previously identified FTA's use of a rigorous and systematic
evaluation process to distinguish among proposed New Starts investments
as a model for other transportation programs.[Footnote 3] However, we
and others have also identified challenges facing the New Starts
program. For example, our past reviews found that many program
stakeholders thought that FTA's process for evaluating New Starts
projects was too complex and costly and did not effectively use all of
the criteria outlined in SAFETEA-LU and previous legislation to account
for different project benefits, such as economic development. This
latter issue is of particular concern, given that FTA's evaluation
process is intended to provide a meaningful and transparent approach
for distinguishing between proposed projects by assessing a range of
project benefits. As a result, by not measuring or underestimating
certain benefits, the relative rankings of proposed projects could
change and subsequently impact FTA's funding recommendations.
In August 2007, FTA issued a Notice of Proposed Rulemaking (NPRM)
[Footnote 4] to implement SAFETEA-LU provisions into the evaluation
process and make additional changes that FTA believes will improve the
New Starts program. However, FTA's proposed changes to the current
evaluation framework were not well received by Members of Congress and
the transit industry, and the Consolidated Appropriations Act of 2008
prohibited FTA from spending money to issue the final rule this fiscal
year.[Footnote 5] These issues and the upcoming reauthorization of all
surface transportation programs, including the New Starts program, have
led stakeholders and policymakers to re-examine the existing evaluation
process and consider potential modifications and other options for
evaluating New Starts projects in the future.
We are required by SAFETEA-LU to report each year on FTA's processes
and procedures for evaluating, rating, and recommending New Starts
projects for federal funding and on FTA's implementation of these
processes and procedures. This report discusses the (1) information
captured by New Starts project justification criteria, (2) challenges
FTA faces as it works to improve the New Starts program, and (3)
options for evaluating New Starts projects. In addition, appendix I
contains an overview of FTA's fiscal year 2009 New Starts Annual Report
and budget request. To address these objectives, we reviewed SAFETEA-
LU, FTA guidance and regulations governing the New Starts program and
other FTA documents, including the annual New Starts report; reviewed
and summarized research about the impacts of transit projects; attended
New Starts Listening Sessions in Washington, D.C. and Charlotte, N.C.
to learn more about the NPRM; interviewed experts, consultants, project
sponsors, industry associations, and Department of Transportation (DOT)
officials about the current and proposed New Starts evaluation
frameworks, as well as other options for evaluating projects; and
analyzed a sample of comments to FTA's docket on the NPRM for New
Starts and Small Starts. Appendix II contains additional information
about our scope and methodology. We conducted this performance audit
from October 2007 to June 2008 in accordance with generally accepted
government auditing standards. Those standards require that we plan and
perform the audit to obtain sufficient, appropriate evidence to provide
a reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives.
Results in Brief:
FTA primarily uses cost-effectiveness and land use criteria to evaluate
New Starts projects, but concerns have been raised about the extent to
which the measures for these criteria capture total project benefits.
To assess the land use criterion, FTA uses three evaluation measures,
including land use in the project area, the extent to which the area
has transit supportive plans and policies, and the performance and
impacts of these policies. FTA's current transportation system user
benefits (TSUB) measure, which is used along with costs to assess a
project's cost-effectiveness, focuses on how proposed projects will
improve mobility by reducing the real and perceived cost of travel. FTA
told us that such mobility improvements are a critical goal of all
transit projects and that most secondary project benefits, including
economic development, are derived from improvements that reduce users'
travel times. While the literature and most experts we consulted with
generally agree with this contention, they also raised concerns that
certain benefits are not captured by the existing cost-effectiveness
measure. For example, experts and project sponsors we spoke to noted
that FTA's TSUB measure does not account for benefits to nontransit
users, such as highway users, or capture any economic development
benefits that are not directly correlated to mobility improvements,
such as benefits to people who are willing to pay more to live near
transit stations in order to preserve their option to use it in the
future.[Footnote 6] As a result, FTA may be underestimating transit
projects' total benefits, particularly in areas looking to use these
projects as a way to relieve congestion or promote more high-density
development. In these cases, it is unclear the extent to which FTA's
current approach to estimating benefits impacts how projects are ranked
in FTA's evaluation and ratings process. FTA officials acknowledged
many of these limitations. However, they also noted that resolving
these issues would be difficult without a substantial investment of
resources to improve and update local travel models, particularly since
these models generate the travel forecasts required to calculate TSUB
and estimate other project benefits.
FTA faces several systemic challenges to improving the New Starts
program, including multiple program goals that are reflected in the
evaluation criteria, limitations in travel modeling capacity, the need
to maintain the rigor while minimizing the complexity of the evaluation
process, and developing clear and consistent guidance for incorporating
qualitative information into the evaluation process. The New Starts
evaluation criteria, which have been delineated in previous
transportation legislation and recently were augmented by SAFETEA-LU to
include economic development, establish multiple goals for the program.
The establishment of multiple goals has led to varying expectations
between FTA and project sponsors about what types of projects should be
funded through the program. For example, experts and project sponsors
told us that transit projects may emphasize multiple goals, including
economic development and mobility improvements, while FTA told us that
the primary emphasis of the New Starts program is to fund transit
projects that create significant mobility improvements and has designed
the evaluation framework to reflect this goal. As a result, some
project sponsors may be devoting substantial resources to apply for New
Starts funding for projects that are incompatible with FTA's
interpretation of the program goals and, thus, will not rate well under
FTA's current evaluation process because they do not seek to achieve
substantial travel time savings. Additionally, models used to generate
local travel demand forecasts are limited. This affects a model's
ability to accurately represent travel behavior, and as a result,
current models may not provide all of the information needed to
properly evaluate transit projects. FTA has taken some steps to
mitigate the modeling limitations, such as incorporating proxy measures
to account for project impacts like land use and developing a request
for proposals to seek approaches for predicting changes in highway user
benefits, but faces challenges in doing so. The Federal Highway
Administration (FHWA) declined to be involved in the request for
proposals because it deemed the issue to be only relevant to transit,
although FTA officials stated that travel model improvements would
affect how all planning is done and, thus, have impacts on numerous
local, state, and federal programs, including highway programs.
Furthermore, they also noted that the request for proposal is only a
first step to improving local travel models, and additional resources
are needed to ensure that these changes can be implemented in the
future. The upcoming reauthorization of all transportation programs,
including the New Starts program, provides an opportunity to seek
additional resources to improve local travel models. Finally, experts
and some project sponsors we spoke with support FTA's rigorous process
for evaluating proposed transit projects but are concerned that the
process has become too burdensome and complex. In response to such
concerns, FTA has tried to simplify and balance the evaluation process
in several ways, including developing the Very Small Starts eligibility
category within the Small Starts program[Footnote 7] and incorporating
qualitative information into its assessments. However, project sponsors
we spoke to emphasized the continued need for clear, consistent
guidance on how such qualitative information will be used.
Experts and project sponsors we interviewed discussed different options
for evaluating proposed transit projects, but identified significant
limitations of each option. Furthermore, all of these options are
impacted by the systemic challenges discussed above, including
limitations of local travel models and the need to balance the rigor of
the evaluation process with minimizing its complexity. The options
identified by experts and project sponsors include the following:
* One option is to revise the current New Starts evaluation process in
order to improve the program and respond to SAFETEA-LU provisions, as
proposed by FTA in its August 2007 NPRM and proposed policy guidance.
While some experts we spoke to appreciated FTA's efforts to maintain
the rigor of the current evaluation process, others noted that the
proposed revisions outlined by the NPRM may still underestimate total
project benefits. For example, FTA's measure of mobility improvements
does not account for benefits accruing to highway users, and its
measures of environmental benefits may not properly distinguish among
projects. FTA acknowledged that some benefits may not be captured by
their proposed measures and told us that they hope to resolve these
issues through collaborative efforts to improve local travel models and
measures of environmental benefits. In particular, FTA officials are
working with officials from the Office of the Secretary on a request
for proposals that would identify ways to better estimate highway
speeds, which could improve the accuracy of local travel models. FTA
also plans to initiate a long-term effort, in consultation with the
transit community, to develop more robust environmental measures.
However, FTA has not yet set timelines for completing these efforts.
Until this latter effort is completed, project sponsors will continue
to develop and submit information on environmental benefits that is not
useful for evaluation and rating purposes.
* A second option is using benefit-cost analysis to evaluate projects.
Unlike FTA's current measures, benefit-cost analysis would attempt to
monetize all benefits and costs, which experts told GAO would be a more
comprehensive approach. While many experts we spoke to said that
benefit-cost analysis is a useful tool for comparing projects' benefits
and costs over time, others noted the difficulty of quantifying certain
benefits, particularly given limitations of local travel models. FTA
officials told us that they do not support using benefit-cost analysis
because of these challenges. In addition, FTA is currently prohibited
by statute from considering the dollar value of mobility improvements
in evaluating projects.[Footnote 8]
* A third option is evaluating projects differently based on their
primary goal, so that federal transit investments better support local
transit goals. However, many experts and project sponsors said that New
Starts projects should go through an evaluation process designed to
evaluate projects on the basis of national priorities.
* A fourth option is devolving the evaluation process to the state
level by making New Starts a formula grant program. Under this
framework, though, the ability of the federal government to influence
and hold projects accountable could be limited.
To improve the New Starts evaluation process and the measures of
project benefits, which could change the relative ranking of projects,
we are recommending that the Secretary of Transportation take the
following five actions: (1) seek additional resources to improve local
travel models in the next authorizing legislation; (2) legislative
change to allow FTA to consider the dollar value of mobility
improvements in evaluating projects, developing regulations, or
carrying out any other duties; (3) direct the Administrator of FTA to
establish a timeline for issuing, awarding, and implementing the result
of its request for proposals on short-and long-term approaches to
measuring highway user benefits from transit improvements; (4) direct
the Administrator of FTA to establish a timeline for completing its
longer term effort to develop more robust measures of transit projects'
environmental benefits that are practically useful in distinguishing
among proposed projects including consultation with the transit
community; and (5) direct the Administrators of FTA and FHWA to
collaborate to improve the consistency and reliability of local travel
models, including the aforementioned request for proposals on
approaches to measuring highway user benefits.
We provided a draft of this report to DOT for review and comment. DOT
generally agreed with the findings and recommendations in this report,
and provided clarifying comments and technical corrections, which we
incorporated, as appropriate.
Background:
FTA generally funds New Starts projects through FFGAs, which are
required by statute to establish the terms and conditions for federal
participation in a New Starts project. FFGAs may also define a
project's scope, including the length of the system and the number of
stations; its schedule, including the date when the system is expected
to open for service; and its cost. For projects to obtain FFGAs, New
Starts projects must emerge from a regional, multimodal transportation
planning process. The first two phases of the New Starts process--
systems planning and alternatives analysis--address this requirement.
The systems planning phase identifies the transportation needs of a
region, while the alternatives analysis phase provides information on
the benefits, costs, and impacts of different options, such as rail
lines or bus routes, in a specific corridor versus a region. The
alternatives analysis phase results in the selection of a locally
preferred alternative, which is the New Starts project that FTA
evaluates for funding. After a locally preferred alternative is
selected, the project sponsor submits an application to FTA for the
project to enter the preliminary engineering phase.[Footnote 9] When
this phase is completed and federal environmental requirements are
satisfied, FTA may approve the project's advancement into final design,
[Footnote 10] after which FTA may approve the project for an FFGA and
proceed to construction. FTA oversees grantees' management of projects
from the preliminary engineering phase through the construction phase.
To help inform administration and congressional decisions about which
projects should receive federal funds, FTA currently distinguishes
between proposed projects by evaluating and assigning ratings to
various statutory evaluation criteria--including both project
justification and local financial commitment criteria--and then
assigning an overall project rating.[Footnote 11] (See fig. 1.) These
evaluation criteria reflect a broad range of benefits and effects of
the proposed project, such as cost-effectiveness, as well as the
ability of the project sponsor to fund the project and finance the
continued operation of its transit system. FTA has developed specific
measures for each of the criteria outlined in the statute. On the basis
of these measures, FTA assigns the proposed project a rating for each
criterion and then assigns a summary rating for local financial
commitment and project justification. These two ratings are averaged
together, and then FTA assigns projects a "high," "medium-high,"
"medium," "medium-low," or "low" overall rating, which is used to rank
projects and determine what projects are recommended for funding.
Projects are rated at several points during the New Starts process--as
part of the evaluation for entry into the preliminary engineering and
the final design phases, and yearly for inclusion in the New Starts
Annual Report. As required by SAFETEA-LU, the administration uses the
FTA evaluation and rating process, along with the phase of development
of New Starts projects, to decide which projects to recommend to
Congress for funding.[Footnote 12] Although many projects receive a
summary rating that would make them eligible for an FFGA, only a few
are proposed for an FFGA in a given fiscal year. FTA proposes FFGAs for
those projects that are projected to meet the following conditions
during the fiscal year for which funding is proposed:
* All nonfederal project funding must be committed and available for
the project.
* The project must be in the final design phase and have progressed far
enough for uncertainties about costs, benefits, and impacts (e.g.,
financial or environmental) to be minimized.
* The project must meet FTA's tests for readiness and technical
capacity, which confirm that there are no remaining cost, project
scope, or local financial commitment issues.
Figure 1: FTA's Current New Starts Evaluation Process:
[See PDF for image]
The figure is an illustration of FTA's Current New Starts Evaluation
Process, as follows:
Overall project rating:
* Local financial commitment summary rating[A];
- Capital finance plan (50% weight);
- Operating finance plan (30% weight);
- Non-New Starts share (20% weight);
* Project justification summary rating[A];
- Cost-effectiveness (50% weight);
- Land use (50% weight);
- Environmental benefits (0% weight)[B];
- Mobility improvements (0% weight)[B];
- Other factors (0% weight)[B].
Source: GAO analysis of FTA data.
[A] The overall project rating is determined by averaging the rating
for project justification and local financial commitment, each of which
is assigned a 50 percent weight.
[B] According to FTA's July 2007 policy guidance on New Starts, these
criteria are not assigned a weight in the evaluation framework. For
more information on how FTA measures and uses these criteria in the
ratings process, see table 1 of this report.
[End of figure]
SAFETEA-LU introduced a number of changes to the New Starts program,
including some that affect the evaluation and rating process.[Footnote
13] For example, given past concerns that the evaluation process did
not account for a project's impact on economic development and FTA's
lack of communication to sponsors about upcoming changes, the statute
added economic development to the list of project justification
criteria that FTA must use to evaluate and rate New Starts projects,
and requires FTA to issue notice and guidance each time significant
changes are made to the program.[Footnote 14] SAFETEA-LU also
established the Small Starts program, a new capital investment grant
program, simplifying the requirements imposed for those seeking funding
for lower-cost projects such as bus rapid transit, streetcar, and
commuter rail projects.[Footnote 15] This program is intended to
advance smaller-scale projects through an expedited and streamlined
evaluation and rating process. FTA also subsequently introduced a
separate eligibility category within the Small Starts program for "Very
Small Starts" projects.[Footnote 16] Small Starts projects that qualify
as Very Small Starts are simple, low-cost projects that FTA has
determined qualify for a simplified evaluation and rating process.
In addition to implementing the Small Starts program, FTA has taken
other steps to implement SAFETEA-LU changes to the New Starts
evaluation process. For example, FTA incorporated economic development
into the existing evaluation framework by considering the information
provided by project sponsors as an "other factor." FTA also sought
public comments on different proposals for revising the evaluation
process to better reflect the statute through the Advanced Notice of
Proposed Rulemaking (ANPRM) and the final NPRM for the New Starts and
Small Starts programs. However, following concerns voiced by Members of
Congress and the transit industry about the weights placed on different
project benefits, FTA was prohibited from using funds to proceed with
the rulemaking process, with the exception of reviewing comments, under
the fiscal year Consolidated Appropriations Act of 2008. Figure 2 shows
a timeline of FTA's efforts to date to implement SAFETEA-LU changes to
the New Starts evaluation and ratings process.
Figure 2: Timeline of FTA's Implementation of SAFETEA-LU Changes:
[See PDF for image]
This figure is an illustration of the timeline of FTA's implementation
of SAFETEA-LU changes, as follows:
August 2005: SAFETEA-LU, which reauthorized the New Starts program and
required changes to the evaluation and ratings process, is signed into
law.
January 2006: FTA issued Guidance on New Starts Policies and
Procedures, which proposes a number of procedural changes and
identifies longer-term changes to the New Starts program. FTA released
the ANPRM for the Small Starts program.
February and March 2006: FTA conducted three listening sessions to
discuss with project sponsors the changes proposed in its January
guidance on New Starts and the ANPRM for Small Starts. FTA‘s docket for
public comment on the proposed procedural changes and long-term changes
for the New Starts program and the ANPRM for the Small Starts program
closed in mid-March.
May 2006: FTA issued guidance on the New Starts fiscal year 2008
evaluation cycle.
June 2006: FTA issued interim guidance on the Small Starts program for
public comment and held two listening sessions to discuss these
proposals with project sponsors.
July 2006: FTA reviewed comments and issued final interim guidance on
Small Starts for the fiscal year 2007 budget cycle.
August 2006: New Starts project applications for the fiscal year 2008
evaluation cycle were due.
August 2007: FTA released the final NPRM for New Starts and Small
Starts to the public for comment and closed the docket in November.
Source: GAO analysis.
[End of figure]
FTA's Project Evaluation Measures Include a Range of Information, but
Not All Project Benefits Are Fully Captured:
FTA primarily uses the cost-effectiveness and land use criteria to
evaluate New Starts projects, but concerns have been raised about the
extent to which the measures for these criteria capture total project
benefits. Specifically, FTA's TSUB measure considers how the mobility
improvements from a proposed project will reduce users' travel times.
According to FTA officials, experts, and the literature we consulted,
the TSUB measure accounts for most secondary project benefits,
including economic development, because these benefits are typically
derived from mobility improvements that reduce users' travel times.
However, project sponsors and experts raised concerns about how FTA
currently measures and weights different project justification
criteria, noting that these practices may underestimate some project
benefits. For example, some experts and project sponsors we spoke to
said that the TSUB measure does not account for benefits for nontransit
users or capture any economic development benefits that are not
directly correlated to mobility improvements. As a result, FTA may be
underestimating projects' total benefits, particularly in areas looking
to use these projects as a way to relieve congestion or promote more
high-density development. In these cases, it is unclear the extent to
which FTA's current approach to estimating benefits impacts how
projects are ranked in FTA's evaluation and ratings process. FTA
officials acknowledged these limitations, but noted that improvements
in local travel models are needed to resolve some of these issues.
FTA Emphasizes Cost-Effectiveness and Land Use in Developing Project
Justification Ratings:
FTA currently relies on the cost-effectiveness and land use criteria to
evaluate and rate New Starts projects.[Footnote 17] Specifically, FTA
assigns a weight of 50 percent to both the cost-effectiveness and land
use criteria when developing project justification ratings. Table 1
provides a summary of all project justification criteria that FTA is
required to review, the measures it uses to evaluate these criteria,
and how this information is used to rate projects.
Table 1: FTA's Current Project Justification Criteria and Measures for
Evaluating and Rating New Starts Projects:
Criterion: Cost-effectiveness[A];
Information evaluated:
* Incremental annualized capital and operating costs of the transit
system with the project;
* Projected transportation system user benefits associated with the
project (including travel time and cost savings, and improvements in
comfort, convenience, and reliability);
Weight: 50%;
How FTA uses this information:
* FTA establishes five breakpoints, each of which reflects a dollar
range for different ratings of a project's cost-effectiveness (i.e.,
high, medium-high, medium, medium-low, and low). FTA assigns a cost-
effectiveness rating to each project, and annually updates these
breakpoints to reflect inflation;
* Proposed projects with a lower cost per hour of projected user
benefits are deemed more cost effective than those with a higher cost
per hour of projected user benefits. Projects generally must receive a
medium or higher cost-effectiveness rating to be recommended for
funding.
Criterion: Land use;
Information evaluated:
* Existing land use;
* Transit supportive plans and policies;
- Growth management;
- Transit supportive corridor policies;
- Supportive zoning regulations near stations;
- Tools to implement land use policies;
* Performance and impact of policies;
- Performance of land use policies;
- Potential impact of transit project on regional land use;
Weight: 50%;
How FTA uses this information:
* FTA evaluates existing land use, transit supportive plans and
policies, and performance and impact of policies by the factors noted
under each category. Projects receive a numerical rating (1 to 5) for
each of these factors, and then these individual factor ratings are
averaged to determine a category-specific rating. FTA then combines
these category-specific ratings into a descriptive rating on FTA's five-
level scale (i.e., high, medium-high, medium, medium-low, and low) to
determine the overall land use rating;
* In rare cases, when based on unusually compelling "other" land use
considerations, FTA may increase the land use rating by one point.
Criterion: Environmental benefits;
Information evaluated:
* Environmental Protection Agency air quality designation;
Weight: 0%[B];
How FTA uses this information:
* FTA does not explicitly weight this measure in the framework because
the measure does not meaningfully distinguish among projects. As a
result, projects receive ratings based on the following:;
* Projects in nonattainment areas[C] for any transportation-related
pollutants receive a high rating;
* Projects that are in attainment areas[C] receive a medium rating.
Criterion: Mobility improvements;
Information evaluated:
* Projected user benefits per passenger mile of the project;
* Projected number of transit dependents using the project;
* Projected user benefits for transit dependents per passenger mile of
the project;
* Projected share of user benefits received by transit dependents
compared to share of transit dependents in the region;
Weight: 0%[B];
How FTA uses this information:
* FTA evaluates projected user benefits per passenger mile of the
project; projected number of transit dependents using the project;
projected user benefits for transit dependents per passenger mile of
the project; and projected share of user benefits received by transit
dependents compared to share of transit dependents in the region.
Projects receive a numerical rating (1 to 5) for each of these
measures. FTA then develops the mobility rating by averaging the rating
for the first measure (which applies to all riders of the New Starts
project) and the combined ratings for the subsequent three (that apply
only to transit dependents);
* FTA does not use the rating for this criterion in determining the
project justification rating, except in certain cases as a tiebreaker
when the average of the cost-effectiveness and land use ratings falls
equally between two categories (e.g., medium and medium-high). The
project rating is "rounded up" unless mobility improvements are rated
low.
Criterion: Other factors;
Information evaluated:
* Project's effect on economic development[D];
* Nature and extent of the transportation problem or opportunity in the
project corridor, as described in the make-the-case document[E];
* Extent to which the project is a principal element of a congestion
management strategy, in general, and a pricing strategy, in
particular[D];
* Any other factor that articulates project benefits but is not
captured within the criteria[D];
Weight: 0%[B];
How FTA uses this information: Each factor will be considered based on
different criteria, and then the rating will be introduced after the
initial project justification rating is determined. If the "other
factors" rating is higher than the initial project justification
rating, FTA may increase this initial justification rating by a maximum
of one step (e.g., from medium to medium-high). If it is lower, FTA may
decrease this initial rating.
Source: GAO summary of FTA guidance.
[A] In FTA's most recent guidance on New Starts, FTA asserted that the
cost-effectiveness criterion captures operating efficiencies, and as a
result, this criterion is no longer evaluated separately.
[B] According to FTA's July 2007 policy guidance on New Starts, these
criteria are not assigned a weight in the evaluation framework. (See
FTA's July 2007 guidance on the fiscal year 2009 New Starts evaluation
and ratings process for more information.)
[C] Nonattainment areas refer to areas of the country where air
pollution levels persistently exceed the national ambient air quality
standards, whereas attainment areas are areas that meet the ambient air
quality standards for the pollutant.
[D] Rating of these factors can only positively affect the project
justification rating, as the absence of a strategy has no effect on the
project justification rating.
[E] The make-the-case document provides sponsors with the opportunity
to discuss the merits of their projects in an essay form and present
additional information not captured by the evaluation process. A high
rating for the make-the-case document may result in an increase in the
overall project justification rating. A low rating for the make-the-
case document may reduce the overall project justification rating.
[End of table]
To evaluate the land use criterion, FTA has developed and uses three
qualitative land use measures: land use in the project area, the extent
to which the area has transit supportive plans and policies, and the
performance and impacts of these policies. For example, to determine
whether a project's surrounding area has transit supportive plans and
policies, FTA examines whether there are growth management strategies
and transit supportive corridor policies in place, the extent to which
zoning regulations near stations are transit supportive, and the tools
available to implement land use policies.
To evaluate cost-effectiveness, FTA relies on the TSUB measure and
costs. The TSUB measure captures predicted improvements in mobility
caused by the implementation of a project. In particular, TSUB captures
transit users' cost and travel time savings, as well as improvements in
comfort, convenience, and reliability of travel. Project sponsors use
local travel models to forecast ridership and simulate trips taken in
2030, the forecast year used to estimate savings over time for two
alternatives.[Footnote 18] To evaluate the benefits for these two
alternatives, FTA uses the outputs from these models to consider and
weigh a range of attributes, such as time spent waiting at and walking
to the transit station, and calculates the perceived level of time
savings associated with a given project. The first alternative, known
as the baseline alternative, assumes low-cost improvements to the
project area's current transportation network, while the second
alternative---the "build alternative"--assumes the proposed New Starts
transit project is constructed. As outlined in figure 3, FTA uses the
forecasts for these two alternatives to calculate the predicted TSUB
value for the proposed project.[Footnote 19] To determine a project's
final cost-effectiveness rating, FTA divides the project's annual
capital and operating costs by its predicted TSUB value and compares
the computed figure to established cost-effectiveness breakpoints.
[Footnote 20]
Figure 3: Example of a TSUB Calculation:
[See PDF for image]
This figure is an illustration of a TSUB calculation, as follows:
Baseline alternative[A]:
100 transit travelers(A):
* Wait time: 5 minutes;
* Local Bus: 20 minutes;
* Transfer time: 4 minutes [4 minutes transfer time + 5 minutes wait
time = 9 minutes total wait/transfer time (baseline);
* Train: 20 minutes;
* arrive at destination [29 minutes train run time + 20 minutes bus run
time = 40 minutes total run time (baseline).
Build alternative[B]:
120 transit travelers [120 travelers (build) - 100 travelers (baseline)
= 20 new travelers(B):
* Wait time: 3 minutes [9 minutes total wait/transfer time (baseline) -
3 minutes total wait/transfer time (build) = 6 minutes wait time
savings (C);
* Train: 30 minutes [40 minutes total run time (baseline) - 30 minutes
total run time (build) = 10 minutes run time savings (D).
User benefits:
Original trips = (A) [100 original travelers] x (2* x (C) [6 min. wait
time savings + (D) [10 min run time savings) + New trips [(B) [20 new
travelers] x (2* x (C) [6 min. wait time savings) divided by 2** =
2,420 minutes.
TSUB value = 2,420 minutes.
* Wait/transfer time savings weighted as double because wait and
transfer times are generally perceived as more onerous by travelers.
** FTA divides the total time savings accruing to new transit users by
two because travelers value the user benefits created by projects
differently[C].
Source: GAO and FTA.
[A] The baseline alternative assumes low-cost improvements are made to
the transportation network.
[B] The build alternative assumes the proposed New Starts project
(i.e., fixed guideway transit infrastructure investment) is
constructed.
[C] For more information on how TSUB is calculated, including why user
benefits are valued differently for new transit travelers, see appendix
III.
[End of figure]
FTA officials that we interviewed noted that the TSUB measure used to
assess the cost-effectiveness criterion in the New Starts evaluation
framework emphasizes predicted mobility improvements because most
project benefits are realized only when transit users perceive that
their time and cost of travel has been reduced. For example, the
introduction of new transit service may reduce users' overall travel
time to a given destination. These reductions in travel time usually
occur because a project offers faster travel times as a result of
travel on the project's fixed guideway, which does not incur the degree
of congestion faced by buses operating in mixed travel. According to
FTA, such transit user benefits are the distinct and primary benefit of
transit investments. Most other benefits of transit projects, such as
economic development, are considered secondary benefits because they
are still directly related to mobility improvements. For example,
transportation investments that improve the accessibility and
attractiveness of certain locations can result in higher property
values in those areas, which can affect the type and density of
development that occurs in the area of the investment. The
transportation literature and different experts we consulted agreed
that such increases in property values are generally the result of
mobility improvements. As such, they noted that conducting a separate
evaluation of secondary benefits, such as economic development, may be
inappropriate because it can result in double counting certain project
impacts. For example, in a 2002 report, the Transportation Research
Board (TRB) reported that secondary benefits like economic development
"are double counts" of mobility improvements and must be carefully
measured and presented "in such a way that decision makers are aware of
the potential for double counting."[Footnote 21]
FTA also considers information on environmental benefits,[Footnote 22]
mobility improvements, and other factors (including economic
development), but these criteria are not weighted in the current
evaluation framework. As a result, they are not used to calculate the
project justification rating, except under certain circumstances. For
example, FTA currently evaluates information on mobility improvements,
but this criterion is not used in determining the project justification
rating, except in certain cases as a tiebreaker when the average of the
cost-effectiveness and land use ratings falls equally between two
categories.[Footnote 23]
Experts and Other Program Stakeholders Expressed Concerns That FTA's
Current Evaluation Measures Could Be Underestimating Total Project
Benefits:
Project sponsors and experts we interviewed raised concerns about how
FTA uses and measures different New Starts project justification
criteria in the evaluation framework, which could potentially result in
certain project benefits being underestimated. Some project sponsors we
spoke with expressed frustration that FTA does not include certain
criteria in the initial calculation of project ratings, such as
economic development and environmental benefits. They noted that this
practice limits the information captured on projects, particularly
since these are important benefits of transit projects at the local
level and were required to be evaluated under SAFETEA-LU. In addition
to these concerns, we have previously reported that FTA's reliance on
two evaluation criteria to calculate a project's overall rating is not
aligned with the multiple-measure evaluation and rating process
outlined in statute and current New Starts regulations.[Footnote 24] As
a result, we recommended that FTA improve the measures used to evaluate
New Starts projects or provide a crosswalk in the regulations showing
clear linkages between the criteria in the statute and the criteria
used in the evaluation process. FTA's current guidance on the New
Starts evaluation process states that environmental benefits are not
weighted presently because the current measure does not meaningfully
distinguish among projects. Furthermore, FTA officials we interviewed
told us that they had not yet developed a reliable way to incorporate
economic development into the framework, had not received any
reasonable suggestions for measuring this criterion, or had project
sponsors submit information demonstrating the impacts of their projects
on economic development. Despite these issues, however, they
acknowledged that the current approach for evaluating projects does not
align with SAFETEA-LU and noted that the revised evaluation process
described in the NPRM and proposed policy guidance was developed to
meet these requirements.
Different experts and project sponsors we interviewed also disagreed
with FTA's emphasis on mobility in the cost-effectiveness measure,
noting that it does not account for other important project benefits.
Specifically, experts and project sponsors, as well as members of the
transit industry and DOT officials, stated that FTA's TSUB measure does
not capture the benefits that accrue to highway users as user benefits
when more people switch to the improved transit service and highway
congestion decreases. The omission of these nontransit user benefits
means that the benefits accruing to motorists are not accounted for in
the evaluation process. In cases where a project's predicted impact on
congestion is significant, this omission may lead FTA to underestimate
a project's total user benefits. Given FTA's focus on cost-
effectiveness in the evaluation process, underestimating user benefits
for certain projects could impact the overall project ratings and
change the relative ranking of proposed transit projects.
In response to this issue, FTA officials told us that although the TSUB
measure and existing software have the capacity to capture highway user
benefits, they do not currently accept estimates of nontransit user
benefits because local travel models do not reliably predict changes in
travel speeds resulting from transit investments. Instead, FTA
currently adjusts the cost-effectiveness breakpoints upward, which has
the effect of giving all projects the same credit for highway travel
time savings. As a result, some projects are being credited with
achieving these benefits, even when the project has no impact at all on
highway travel time savings, while other projects may not be receiving
enough credit for their impact on highway travel time savings. FTA
officials noted that they would prefer to estimate the predicted impact
of projects on highway congestion rather than using a rough proxy for
these benefits, particularly since their current approach does not
distinguish among projects in a meaningful way. Officials at FTA and
the Office of the Secretary of Transportation also told us that they
are conducting research on ways to improve the estimation of highway
speeds (and thus, the calculation of nontransit user benefits) by local
travel models, but a significant investment of resources by different
levels of government will likely be required to do so.
A few experts we spoke with also commented that FTA's cost-
effectiveness measure does not capture any project benefits, such as
economic development effects, that are unrelated to mobility
improvements. As noted earlier, FTA contends that its emphasis on
mobility improvements is appropriate, since most secondary project
benefits--including economic development--are derived from this
measure. Although our work, the transportation literature we reviewed,
and experts we consulted generally support this contention, these
sources also indicated that some secondary project benefits, namely
certain economic development effects, may not always accrue in direct
proportion to mobility improvements. Some studies we reviewed and
experts we spoke with noted that property value increases near a
project may occur due to option value or agglomeration effects, both of
which are indirect results of transit investments and not explicitly
related to mobility improvements.[Footnote 25] In such cases, FTA's
existing TSUB measure would understate the total benefits that result
from providing enhanced access to a dense urban core, rather than
transporting commuters from longer distances (e.g., light or heavy
rail) due to its emphasis on travel time savings.[Footnote 26]
Furthermore, our previous work on measuring costs and benefits of
transportation investments has stated that there could be some residual
benefit from these indirect effects that is not accounted for in travel
time benefits or other direct impacts.[Footnote 27] This lack of
accounting for certain secondary benefits in the TSUB measure may
prevent FTA from capturing all project benefits and developing accurate
project rankings.
In interviews with FTA officials about this issue, they acknowledged
that some benefits may accrue in varying proportions to mobility
improvements--that is, certain benefits may not be directly related to
changes in mobility improvements. In such cases, the current evaluation
process may not favor certain types of projects--such as streetcars--
that are not designed to create travel time savings, but rather create
other benefits. Such benefits could include changes in land use that
are not captured by the TSUB measure. In the future, FTA officials told
us that they would prefer to improve local models, so that they can
consistently and reliably assess projects' impact on nontransit users
and economic development.
Finally, some project sponsors also expressed concern about FTA's
requirement to use fixed land use assumptions[Footnote 28] when
estimating the predicted user benefits resulting from the
implementation of a proposed project. According to sponsors, this
practice prevents FTA from explicitly counting some future benefits
that may arise due to an area's increased accessibility. For example,
some transit projects' primary goal is to change land use around
transit stations in order to capitalize on the area's enhanced
accessibility. Such changes could also lead to increases in future
transit ridership, resulting in higher user benefits for the project.
Furthermore, a recent panel of experts convened by FTA noted that it
was unrealistic to evaluate only the incremental impacts of the
proposed transit project, since local governments often find it
difficult to justify high-density, mixed-use zoning in the absence of
transit. Thus, by assuming that no such land use changes will occur,
FTA may be underestimating projects' predicted user benefits.[Footnote
29]
FTA officials told us they have two reasons for fixing land use
assumptions when calculating user benefits. First, it is difficult to
determine the magnitude of the additional land use changes, including
economic development that will result from a project. Most localities
do not have analytical methods for these projections, and the methods
that do exist are often more unreliable than the local models used to
forecast travel demand. Second, even with a reasonable estimate of
additional development, it is difficult to value the benefits of the
additional development. Officials from FTA told us that significant
changes to local travel models would be required before they could
allow project sponsors to vary their assumptions about future land use
when estimating user benefits.
FTA Faces Several Systemic Challenges to Improving the New Starts
Program:
FTA faces several systemic challenges to improving the New Starts
program, including addressing multiple program goals, limitations of
local travel models, the need to maintain the rigor while minimizing
the complexity of the evaluation process, and developing clear and
consistent guidance for incorporating qualitative information into the
evaluation process. FTA and project sponsors we spoke with have
interpreted the emphasis of the New Starts program differently because
the evaluation criteria, which have been delineated in previous and
existing transportation legislation, establish multiple goals for the
program.[Footnote 30] Additionally, models used to generate local
travel demand forecasts have limited capabilities and may not provide
all of the information needed to properly evaluate transit projects.
FTA has taken some steps to mitigate the modeling limitations but faces
challenges in doing so, including a lack of resources to invest in
local travel model improvements. Finally, experts, transportation
consultants, and some project sponsors we spoke with support FTA's
rigorous process for evaluating proposed transit projects but are
concerned that the process has become too burdensome and complex. FTA
has taken some steps to streamline its evaluation process and
incorporate qualitative information into the assessment, but project
sponsors we spoke to emphasized the continued need for clear,
consistent guidance on how such qualitative information will be used.
FTA and Project Sponsors Have Interpreted Emphasis of New Starts
Program Differently:
FTA and project sponsors we spoke with have interpreted the emphasis of
the New Starts program differently. Although the goals have not been
explicitly articulated in legislation, the evaluation criteria outlined
within the law express various goals of the New Starts program. These
include mobility improvements, environmental benefits, operating
efficiencies, cost-effectiveness, economic development, and land use.
The presence of multiple program goals within the statute, as
articulated by the evaluation criteria, has led to different
interpretations by FTA and project sponsors about what project benefits
should be emphasized in the New Starts evaluation process. As noted
earlier, FTA focuses on mobility improvements in its evaluation process
because it contends that those benefits are a critical goal of all
transit projects and that most secondary project benefits, including
economic development, are derived from improvements that reduce users'
travel times. Many of the experts and some of the project sponsors we
spoke to agreed that transit projects can work toward a number of
different goals, including mobility improvements, though some project
sponsors told us that creating nontransportation benefits, such as
generating local economic development, can be the primary goal of a
project. In the latter case, the primary goal of a project is not to
create significant mobility improvements, but rather to stimulate high-
density development and change land use patterns around a transit
station. Accordingly, such projects may not generate the mobility
improvements needed to qualify for New Starts funding under the current
New Starts evaluation process. Some project sponsors, therefore, could
devote substantial resources to apply for New Starts funding for
projects that are incompatible with FTA's emphasis on mobility
improvements.
Local Modeling Limitations Prevent Full Evaluation of Project Impacts:
The models used to generate local travel forecasts are limited and may
not provide sufficient or reliable information to properly evaluate
transit projects. According to a recent report by TRB, the demands on
local models have grown significantly in recent years as a result of
new policy concerns, such as the need to estimate motor vehicle
emissions and evaluate alternative land use policies, and existing
models are inadequate to address many of these new concerns.[Footnote
31] The current models used by most MPOs are generally able to
represent aggregate and corridor-level travel demand, but they are not
dynamic. That is, they are based on average travel speeds over discrete
areas and cannot represent the conditions that would be expected by an
individual traveler choosing how, when, and where to travel. This
limitation affects a model's ability to accurately represent travel
behavior, nonauto (e.g., walking or biking) or transit travel, and
transit's impacts on highway congestion, thereby limiting a model's
ability to provide all of the information needed to properly evaluate
transit projects.
Some of the experts, as well as FTA and Office of the Secretary
officials we interviewed, agreed that local modeling capacity is
limited and should be updated to better reflect travel behavior. For
example, one expert maintained that transit projects' estimated impacts
on all travel in the region can be tested with estimates that are
"sensitive" enough to pick up projects' impacts, but noted that most
MPOs do not have the capacity to generate such estimates. In addition,
the TRB report and some experts we spoke with have expressed concerns
that many MPOs have inadequate traffic and household data to validate
their models and provide information on the travel behavior of
different populations. Our past work has also cited the difficulties of
accurately predicting changes in traveler behavior, land use, or usage
of highways resulting from a transit project with current travel
models, as well as concerns about the quality of data inputs into local
travel models.[Footnote 32]
FTA has taken some steps to mitigate the modeling limitations--which
TRB recognized in its report on the state of the practice--but faces
challenges in doing so. As previously discussed, FTA has developed
proxy measures to account for certain project benefits that cannot be
accurately modeled at the present time, such as projects' impacts on
highway congestion. FTA officials told us that they would prefer to
improve local models so that they can consistently and reliably assess
projects' impacts on nontransit users and economic development. To that
end, FTA has recently developed a request for proposals to seek
approaches for predicting changes in highway user benefits that can be
used in the short-term (within 5 years). However, the request for
proposals has not yet been issued or awarded, and there is no timeline
for doing so. Additionally, according to officials from FTA and the
Office of the Secretary, FTA approached FHWA to help with this effort,
but FHWA declined to be involved because it deemed the issue to be only
relevant to transit.[Footnote 33] As a result, the Office of the
Secretary provided the other half of the funding for the request for
proposals. Officials from FTA and the Office of the Secretary stated
that the improvements to travel models would affect the way all
planning is done and, thus, have impacts on numerous local, state, and
federal programs, including highway programs.
Officials from FTA and the Office of the Secretary emphasized that the
request for proposals is just a small step forward to improve modeling.
In the long-term, larger, more fundamental changes are needed to create
dynamic travel models. For example, current models would need to be
adjusted to capture the movement of individuals rather than parts of
the transportation system, such as a highway segment. Additionally,
models need to be altered so that they produce second-by-second results
rather than results by groups of hours. These long-term improvements
would allow for reliable and accurate estimates of highway user
benefits resulting from transit-related mobility improvements and would
also improve travel speed estimates at both the regional and micro
levels. Like the efforts to improve approaches for predicting changes
in highway user benefits, FTA and Office of the Secretary officials
said that these long-term changes in modeling will benefit many
transportation programs beyond the New Starts program.
However, FTA and Office of the Secretary officials told us that a
significant investment of resources by all levels of government will
likely be required to overcome current modeling limitations. In its
2007 report, TRB called for $20 million annually to update local travel
models across the country. Currently, DOT invests about $2.4 million
annually to improve modeling capabilities. Approximately $500,000 per
year is allocated to DOT's Travel Model Improvement Program, which is
designed to assist MPO model development efforts, and another $1.9
million is set aside annually through SAFETEA-LU for the development of
TRANSIMS.[Footnote 34] TRB also reported that MPOs face similar
challenges. Specifically, MPO budgets for model development have not
grown commensurately with travel modeling and forecasting requirements
at the federal level, and staffing levels often limit the extent to
which MPOs can focus on improvements to travel models in addition to
their typical obligations.
Striking Appropriate Balance between Maintaining a Robust Evaluation
Process and Minimizing the Complexity Is Challenging:
Experts and some project sponsors we spoke with generally support FTA's
quantitatively rigorous process for evaluating proposed transit
projects but are concerned that the process has become too burdensome
and complex, and as noted earlier, may underestimate certain project
benefits. For example, several experts and transportation consultants
told us that although it is appropriate to measure the extent to which
transit projects create primary and secondary benefits, such as
mobility improvements and economic development, it is difficult to
quantify all of these projected benefits. Additionally, several project
sponsors noted that the complexity of the evaluation process can
necessitate hiring consultants to handle the data requests and navigate
the application process--which could increase the project's costs. Our
previous reviews of the New Starts program have noted similar concerns
from project sponsors. For example, in 2007, we reported that a
majority of project sponsors told us that the complexity of the
requirements--such as the analysis and modeling required for travel
forecasts--creates disincentives for entering the New Starts pipeline.
[Footnote 35] Sponsors also said that the expense involved in
fulfilling the application requirements, including the costs of hiring
additional staff and consultants, discourages agencies with fewer
resources from applying for this funding.
In response to such concerns, FTA has tried to simplify the evaluation
process in several ways. For example, following SAFETEA-LU's passage,
FTA established the Very Small Starts eligibility category within the
Small Starts program for projects less than $50 million in total cost.
This program further simplifies the application requirements in place
for the Small Start program, which funds lower-cost projects, such as
bus rapid transit, streetcar, and commuter rail projects. Additionally,
in its New Starts program, FTA no longer rates projects on the
operating efficiencies criterion because, according to FTA, operating
efficiencies are already sufficiently captured in FTA's cost-
effectiveness measures, and the measure did not adequately distinguish
among projects.[Footnote 36] Thus, projects no longer have to submit
information on operating efficiencies. Likewise, FTA no longer requires
project sponsors to submit information on environmental benefits
because it found that the information gathered did not adequately
distinguish among projects and that EPA's ambient air quality rating
was sufficient. FTA also commissioned a study by Deloitte in June 2006
to review the project development process and identify opportunities
for streamlining or simplifying the process.[Footnote 37] This study
identified a number of ways that FTA's project development process
could be streamlined, including revising the policy review and issuance
cycle to minimize major policy and guidance changes to every 2 years
and conducting a human capital assessment to identify skill gaps and
opportunities for reallocating resources in order to enhance FTA's
ability to review and assist New Starts projects in a timely and
efficient manner. FTA is working to implement these recommendations.
Incorporating Qualitative Information into the Evaluation Process Is
Challenging:
Incorporating qualitative information into the New Starts evaluation
process can provide a more balanced approach to evaluating transit
projects, but developing clear and consistent guidance for
incorporating qualitative information can be challenging. Though a
quantitative evaluation process can be both rigorous and transparent,
it does have limitations. Our past work and some experts and project
sponsors we interviewed expressed concern about using a strictly
quantitative process when evaluating proposed transportation
investments because, as discussed above, certain benefits cannot be
easily quantified. For example, some project sponsors and experts said
that because certain impacts, such as economic development, cannot be
easily quantified, a qualitative approach is needed to ensure that
those project impacts are included in the New Starts evaluation
process. Additionally, experts and project sponsors we spoke with
raised concerns about FTA's heavy reliance on quantitative measures in
the New Starts evaluation process, noting that it can be very costly to
run multiple iterations of travel models (which a quantitative-focused
evaluation process requires) and that some transit agencies do not have
the expertise to refine their models to FTA's specifications.
In recognition of the limitations of a quantitative analysis, FTA has
integrated some qualitative information into its current evaluation
process. For example, FTA currently uses three qualitative land use
measures to evaluate a transit project's potential land use impacts.
The NPRM also proposes to incorporate some qualitative information into
the evaluation process, including measures of a transit project's
impact on economic development. Additionally, FTA incorporated the make-
the-case document into its evaluation process in 2003, which allows
project sponsors to submit an essay that justifies why the New Starts
project is the best possible alternative and why it is needed. Although
the fiscal year 2009 rating cycle was the first time that FTA planned
to rate the make-the-case documents for the evaluation process, it
ultimately decided not to because agency officials were generally
dissatisfied with the quality of the make-the-case documents submitted.
[Footnote 38] FTA officials attributed the overall unsatisfactory
quality of the make-the-case documents to insufficient guidance about
what information to include in the document and how this information
would be evaluated. FTA told us that they are working to improve the
guidance for the next rating cycle. According to a few project sponsors
we spoke to, FTA's recent experience with the make- the-case document
illustrates the need for consistent, transparent guidance for using
qualitative information in its evaluation process. To help FTA
incorporate qualitative information into the evaluation and rating
process in a transparent and consistent manner, a few experts we spoke
with suggested that FTA convene an external panel of transportation
experts to rate qualitative information, such as the make-the-case
document and the economic development criterion.
Different Options for Evaluating Proposed New Starts Projects Exist,
but All Have Limitations:
Different options for evaluating proposed transit projects exist.
However, all have limitations and are impacted to varying degrees by
the systemic challenges previously identified, including local modeling
limitations and the need to balance the rigor of the evaluation process
with an interest in minimizing complexity. One option is to revise the
current evaluation process as proposed by FTA in the August 2007 NPRM
and proposed policy guidance. A second option is to use benefit-cost
analysis as the evaluation framework for projects. A third option is to
use evaluation frameworks that vary by project goal in order to better
support local transit priorities. A fourth option is to eliminate the
federal evaluation process and devolve these responsibilities to the
state level by making New Starts a formula grant program.
FTA's Proposed Revisions to Existing Evaluation Process Address Some
Concerns but May Continue to Inaccurately Estimate Total Project
Benefits:
One option to evaluate proposed transit projects is to revise the
existing New Starts evaluation process, as proposed by FTA. In response
to provisions in SAFETEA-LU and to improve the New Starts program, FTA
proposed to revise the current process by introducing new evaluation
measures and weights, as described in its August 2007 NPRM and proposed
policy guidance. The proposed process revises the current evaluation
process to reflect the multiple measure approach to evaluating transit
projects described in SAFETEA-LU. As in the current process, FTA's
proposed evaluation process assigns ratings to projects on the basis of
various evaluation criteria to determine summary ratings for both local
financial commitment and project justification (see fig. 4). In
contrast to the current process, however, the proposed process places
weights on measures that were previously not used to calculate initial
project justification ratings, including environmental benefits,
economic development, and mobility improvements.
Figure 4: FTA's Proposed New Starts Evaluation Process:
[See PDF for image]
This figure is an illustration of FTA's proposed new starts evaluation
process, as follows:
Overall project rating:
* Local financial commitment summary rating[A];
- Capital finance plan (50% weight);
- Operating finance plan (50% weight);
- Non-New Starts share (0% weight)[C];
* Project justification summary rating[B];
- Cost-effectiveness (50% weight) [Cost-effectiveness (50% weight)];
- Effectiveness rating (50% weight) [Mobility improvements (25%
weight)[B]], Economic development/land use (20% weight), Environmental
benefits (5% weight)].
Source: GAO analysis of FTA data.
[A] The overall project rating is determined by averaging the rating
for project justification and local financial commitment, each of which
is assigned a 50 percent weight.
[B] According to FTA's August 2007 Proposed Policy Guidance on New
Starts, this criterion will not be assigned an explicit weight in the
evaluation framework. For more information on how FTA plans to use the
information captured under this criterion in the ratings process, see
the last row of table 2.
[C] If the amount of New Starts funding requested is less than 50% of
the total project cost and the project has an overall local financial
commitment rating of "medium" or "medium-high," the rating would be
increased one level.
[End of figure]
Under the proposed evaluation process, project justification criteria
are grouped into categories of "cost-effectiveness" and
"effectiveness." The cost-effectiveness category accounts for 50
percent of the overall project justification rating and is based on the
current measure of cost-effectiveness with no proposed changes. The
effectiveness category accounts for the other 50 percent of the project
justification rating and is based on measures of (1) mobility
improvements,[Footnote 39] (2) economic development and land use, and
(3) environmental benefits. See table 2 for descriptions of all the
proposed project justification measures.
Table 2: FTA's Proposed Project Justification Measures for Evaluating
and Rating New Starts Projects:
Criterion: Cost-effectiveness[B];
Information evaluated:
* Annualized capital and operating costs of project;
* Projected benefits for users of transit system (including travel time
and cost savings, and improvements in comfort, convenience,
reliability);
Weight: 50%;
How FTA uses this information:
* FTA establishes breakpoints to assign a cost-effectiveness rating to
each project, and annually updates these breakpoints to reflect
inflation;
* Proposed projects with a lower cost per hour of projected user
benefits are deemed more cost-effective than those with a higher cost
per hour of projected user benefits.
Criterion: Land use and economic development;
Information evaluated:
* Current population, employment, and development patterns;
* Development and land use policies and plans;
* [Italics] Population, employment, and property value growth in
project corridor over previous 5 years [End italics];
* [Italics] Projected benefits for users of transit system (including
travel time and cost savings, and improvements in comfort, convenience,
reliability) [End italics];
* [Italics] Value of fixed assets, such as transit stations, in the
corridor divided by the total cost of the proposed project [End
italics];
Weight: 20%;
How FTA uses this information:
* FTA will use a combination of quantitative and qualitative measures
of likely economic development and land use benefits;
* The measures are based on the circumstances in which the projects
would be implemented, such as the strength of the real estate and
employment markets, rather than forecasts of projects' specific impacts
on development and land use patterns because FTA contends that few
appropriate predictive tools are available in standard practice.
Criterion: Environmental benefits;
Information evaluated:
* [Italics] Projected environmental impact of project [End italics];
* [Italics] Proposals for minimizing environmental impact of project
[End italics];
* [Italics] Extent of air pollution in project's service area [End
italics];
Weight: 5%;
How FTA uses this information:
* FTA will give equal weight to the three environmental factors in
determining the overall rating for environmental benefits.
Criterion: Mobility improvements;
Information evaluated:
* Projected user benefits per passenger mile of the project;
* [Italics] Current congestion levels in project corridor [End
italics];
* [Italics] Projected average weekday ridership [End italics];
* Projected user benefits for transit dependents per passenger mile of
the project;
* Projected number of transit dependents using the project;
* Projected share of user benefits received by transit dependents
compared to share of transit dependents in the region;
Weight: 25%;
How FTA uses this information:
* General mobility will be calculated based on three equally weighted
factors: (1) user benefits per passenger mile on the project (as
currently calculated); (2) severity of current congestion in the
project corridor; and (3) average weekday ridership;
* Transit dependent mobility will be calculated based on modified
versions of the three current transit dependent measures, as well as on
the extent to which previous projects in the region have benefited
transit dependents;
* FTA will evaluate projects on the basis of predicted general mobility
benefits (weighted as 20 percent of the overall project justification
rating) and predicted transit dependent mobility benefits (weighted 5
percent).
Criterion: Other factors;
Information evaluated:
* Nature and extent of the transportation problem or opportunity in the
project corridor as described in the make-the-case document;
* Extent to which the project is a principal element of a congestion
management strategy, in general, and a pricing strategy, in particular;
* Any other factor that articulates the benefits of the proposed
project but is not captured within the other criteria;
Weight: 0%[C];
How FTA uses this information:
* FTA will assign a rating of "high," "medium," or "low" to the
strength of the information contained in the make-the-case document.
FTA will use make-the-case ratings of "high" and "low" to determine the
project justification rating of projects that are at the margin between
two overall rating outcomes;
* The project justification summary rating may be increased if a
project is part of a congestion or pricing strategy and the rating is
near a breakpoint. Because the magnitude of the effect is not well
captured by travel forecasts, consideration of pricing strategies under
the general mobility measure allows FTA to account for the expected
increase in transportation benefits, even if they are not readily
verifiable.
Source: GAO analysis of FTA guidance.
Note: Italics indicate new measures introduced in FTA's August 2007
Proposed Policy Guidance.
[A] The weights noted in the table are for the criterion's contribution
to the overall project justification rating and not to the cost-
effectiveness and effectiveness ratings. Projects must achieve a medium
cost-effectiveness rating to be approved, regardless of the ratings for
the other criteria.
[B] The NPRM framework seeks to formalize that the cost-effectiveness
criterion captures operating efficiencies. As a result, the operating
efficiencies criterion is no longer a separate evaluation criterion,
despite 49 U.S.C. § 5309(d)(2)(B).
[C] According to FTA's August 2007 Proposed Policy Guidance, no weight
is assigned to the other factors criterion. However, as described in
the table, information submitted under this criterion can affect the
project justification summary rating.
[End of table]
Experts and Project Sponsors Generally Disagree on Weights Placed on
Project Benefits in Proposal, but Agree That Revisions Preserve Rigor
of Evaluation Process:
Although experts and project sponsors had differing opinions, many
experts we spoke to generally thought that the weights proposed for the
project justification criteria were appropriate. In particular, many
said it was appropriate that FTA retained its emphasis on mobility
improvements in the proposed evaluation framework by weighting the cost-
effectiveness criterion heavily. They generally agreed with FTA's
assumption that societal benefits from transit projects generally
result from user benefits--that is, reductions in the real and
perceived cost of travel. As such, FTA's measure of predicted user
benefits accounts for many project benefits. Under the proposed
process, FTA would measure different dimensions of user benefits as
part of its cost-effectiveness, mobility improvements, and economic
development criteria. In addition, as called for by many project
sponsors and experts we spoke to, the proposed framework places weights
on measures of economic development, environmental benefits, and other
factors, such as congestion impacts. Many of those experts said that
the weights placed on economic development and environmental benefits
are appropriate. In particular, the experts said that the relatively
low weight placed on the measures of economic development is
appropriate because transit-related development benefits are generally
transfers of economic activity from one area to another and not net
benefits to a region. They also said that many economic development
benefits result from user benefits, and as such, they are captured in
the cost-effectiveness criterion. As we have reported in the past,
these benefits represent real benefits for the jurisdiction making the
transportation improvement but are considered transfers and not real
economic benefits from a regional or national perspective.[Footnote 40]
Further, although SAFETEA-LU lists economic development effects and
transit supportive land use as separate project justification criteria,
most of the experts we spoke to agreed with FTA that combining measures
of economic development and land use into a single evaluation criterion
is appropriate because the two criteria are strongly related. Although
many experts generally agreed with the weights proposed, some project
sponsors we spoke to disagreed with the weights placed on the
evaluation criteria. In particular, they told us that transit user
benefits, as measured under the cost-effectiveness and mobility
improvements criteria, continue to be weighted too heavily under the
proposed evaluation process. They stated that mobility improvements are
emphasized at the expense of other project benefits, such as economic
development. A provision in the SAFETEA-LU Technical Corrections Act of
2008[Footnote 41] amended the language of 49 U.S.C.§ 5309 to require
that FTA give comparable, but not necessarily equal, numerical weight
to each project justification criteria in calculating the overall
project rating. This provision could potentially address the foregoing
concerns, as FTA is now required to capture project benefits in a
comparable manner. However, an FTA official told us that the evaluation
process proposed in their August 2007 NPRM and proposed policy guidance
would have made the change now expressed in law by proposing to weight
each of the different criteria included in the statute.
Furthermore, according to experts and project sponsors we spoke with,
the proposed revisions to the current evaluation process preserve the
rigor of FTA's existing evaluation framework. Unlike the Federal Aid
Highway Program, in which funds are automatically distributed to states
via formulas, the New Starts program's evaluation process requires
local transit agencies to compete for project funds based on specific
financial and project justification criteria. As noted by some experts
we spoke with and in our past work, the use of such a rigorous and
systematic evaluation process helps to properly distinguish among
different projects and could serve as a model for other transportation
programs.[Footnote 42] Further, some project sponsors also noted that
use of the make-the-case document, as proposed under the "other
factors" criterion, could be an effective way to incorporate additional
qualitative information into the evaluation process.
Proposed Revisions May Still Inaccurately Estimate Total Project
Benefits because of Modeling Limitations and Use of Proxy Measures:
Although experts and project sponsors had differing opinions, many
experts and project sponsors noted that the revised process may still
inaccurately estimate total project benefits because of how certain
benefits are measured. As a result, without improvements to the way FTA
measures certain project benefits, it risks ranking proposed projects
inaccurately. In particular, some experts and project sponsors we spoke
with expressed continued concern about how FTA measures user benefits
for the purposes of rating projects' cost-effectiveness, noting the
lack of accounting for nontransit user benefits, such as highway users,
and the use of fixed land use assumptions when calculating transit user
benefits. As previously discussed, FTA maintains that its measure of
transit user benefits is the best that can be done given local modeling
limitations and recognizes that these limitations may impact the
relative ranking of proposed projects. Many project sponsors and
experts we spoke to also expressed concern about how FTA measures
project costs when determining the cost-effectiveness rating. As
required by FTA, the cost used for this rating must include "all
essential project elements necessary for completion of the project."
According to FTA, there has been much discussion in the past as to what
constitutes an essential element of the project versus a project
"betterment."[Footnote 43] In its August 2007 NPRM, FTA sought industry
comment on how the concept of essential project elements should be
addressed in the evaluation process. Many of the stakeholders we
consulted, as well as comments submitted to FTA's docket, said that
betterments should be excluded from the project cost when calculating
cost-effectiveness. This could result in better cost-effectiveness
scores for some proposed projects, according to FTA. Some stakeholders
we spoke to also noted that defining what an essential project element
is can be difficult.
Although many experts we spoke to agreed with the weight placed on cost-
effectiveness in the evaluation process, some also said that FTA should
not rely solely on the TSUB measure as a proxy for all other benefits,
which they maintained is the practical effect of both the current and
proposed evaluation processes. Some benefits, such as economic
development unrelated to mobility improvements, are not captured by the
TSUB measure or the proposed new measures of project benefits,
according to many experts we spoke to. FTA's continued emphasis on its
measures of mobility in the revised evaluation process may lead to
underestimating projects' total benefits and, thus, inappropriately
ranking proposed projects. FTA acknowledged this concern in its August
2007 Proposed Policy Guidance, noting that not all transit-related
economic development is the result of improvements in mobility. FTA is
currently studying the magnitude of benefits unrelated to mobility
improvements that result from projects and told us that local modeling
limitations have made it difficult to estimate projects' land use
impacts. In particular, FTA convened an expert panel on October 17,
2007, to discuss methods for evaluating the economic development
benefits of transit projects. FTA's intended objective is to develop,
to the extent possible, a standardized, empirically based, and rational
method for evaluating the potential economic development benefits of
New Starts projects. (See table 3 for more information on the proposed
evaluation measures.)
Table 3: Extent to Which FTA's Proposed Evaluation Measures Address
NPRM Stakeholder Concerns:
Criteria: Cost-effectiveness and mobility improvements;
Concern with current process as expressed by stakeholders in NPRM:
* Does not include nontransit user benefits, such as highway travel
improvements;
FTA response in NPRM:
* Incorporates a measure of current congestion levels in the project
corridor as a proxy for highway user benefits;
FTA explanation:
* FTA does not use estimates of nontransit user benefits because local
travel models do not reliably predict changes in highway travel speeds;
FTA officials acknowledged that the congestion benefits measure is an
imperfect proxy but is appropriate given modeling limitations.
Criteria: Cost-effectiveness and mobility improvements;
Concern with current process as expressed by stakeholders in NPRM:
* Does not capture benefits that do not accrue in proportion to
mobility improvements, such as economic development impacts;
FTA response in NPRM:
* Incorporates new measures of economic development;
FTA explanation:
* FTA acknowledges that there may be variation among projects in the
extent to which other benefits accrue in proportion to mobility
improvements.
Criteria: Cost-effectiveness and mobility improvements;
Concern with current process as expressed by stakeholders in NPRM:
* Does not allow for varying land use assumptions over time when
calculating transit user benefits;
FTA response in NPRM:
* None;
FTA explanation:
* FTA fixes current land use patterns when calculating user benefits
because it is difficult to determine which land use changes are
appropriate to allow and local land use models are not reliable.
Criteria: Land use;
Concern with current process as expressed by stakeholders in NPRM:
* Does not capture economic development benefits of projects;
FTA response in NPRM:
* Incorporates new measures of economic development;
FTA explanation:
* FTA's proposed measures are based on current conditions, rather than
forecasts of projects' impacts, because FTA contends that few
predictive tools are available in standard practice.
Criteria: Environmental benefits;
Concern with current process as expressed by stakeholders in NPRM:
* Does not capture predicted project impacts on air quality and
greenhouse gas emissions;
FTA response in NPRM:
* None;
FTA explanation:
* Measures of the predicted impacts on air quality and greenhouse gas
emissions have not been proposed in order to avoid placing additional
burden on project sponsors.
Source: GAO analysis.
[End of table]
Some experts and project sponsors also expressed concern that the
proposed evaluation process introduces evaluation measures that will
not appropriately distinguish among projects. In particular, they said
that FTA's proposed measures of economic development, congestion, and
environmental benefits are crude proxy measures of the real benefits
and will not meaningfully distinguish among projects. FTA officials
acknowledged that the proposed measures of environmental benefits are
imperfect proxies but said that they are the most appropriate measures
available to distinguish among projects, given the difficulties in
forecasting the impact of projects on the environment. Further, they
said that they decided not to propose measures of the predicted impact
of projects on the environment, including air quality and greenhouse
gas emissions, in order to avoid placing additional burden on project
sponsors. The officials also said that they are conducting research to
identify other technically appropriate measures. In particular, FTA's
August 2007 Proposed Policy Guidance states that the agency is
initiating a long-term effort, in consultation with the transit
community and environmental experts, to develop more robust
environmental measures that will be effective at distinguishing among
candidate projects. However, FTA has not established a timeline for
this effort and, according to transit associations we spoke with, has
not contacted them to publicize this long-term project. FTA officials
also acknowledged that the proposed measure of congestion impacts, as
part of the mobility improvements criterion, is an imperfect proxy, but
is appropriate given difficulties in forecasting the impact of projects
on nontransit users. Also, as noted earlier, FTA is collaborating with
the Office of the Secretary to develop methods of measuring transit's
impact on highway users. Given local travel modeling limitations and
SAFETEA-LU provisions, FTA officials told us that their proposed
measures of congestion and environmental benefits are appropriate,
respond to the intent of SAFETEA-LU, and minimize the burden on project
sponsors. However, some experts and project sponsors told us that these
proxy measures make the evaluation process more complicated without
improving the relative ranking of projects. To appropriately balance
the rigorous evaluation of projects with the complexity of the process,
many experts and project sponsors said that FTA should include only
those evaluation measures that help properly distinguish among
projects.
Furthermore, some experts and project sponsors we spoke with said FTA's
proposed measures of economic development are not appropriate because
they will not capture projects' impacts on local development patterns.
They noted that the measures should be of predicted impacts and not of
current conditions. Because local models do not reliably predict the
complex interaction between transit projects and land use, some experts
and project sponsors we spoke to said that FTA should rely on both
quantitative and qualitative measures to evaluate projects' predicted
economic development impacts. For example, a project sponsor told us
that local economic models along with surveys of local real estate
experts can be used to help assess the future impact of a transit
project on a corridor's development. FTA officials told us that the
proposed measures of economic development and land use are drawn from
research identifying the causal factors for economic development and
therefore are the most appropriate and reliable measures available
given difficulties in forecasting the impact of transit projects on
economic development and land use. FTA officials also noted that they
have solicited feedback about measuring these benefits in the past and
have not received any practical or appropriate suggestions.
Benefit-Cost Analysis Is Another Evaluation Option, though
Implementation Challenges Exist:
A second option to evaluate proposed transit projects is benefit-cost
analysis. Benefit-cost analysis, a process that attempts to quantify
and monetize benefits and costs accruing to society from an investment,
can be used to identify investment alternatives with the greatest net
benefit to the locality, region, or nation. This analysis examines the
immediate and long-term effects of the investment for both users and
nonusers. Because benefit-cost analysis can be used to systematically
assess proposed investments, it may be a useful tool for evaluating New
Starts projects. Although using this approach to evaluate other federal
investments is commonly advocated, FTA is currently prohibited from
considering the dollar value of mobility improvements in evaluating
projects, developing regulations, or carrying out any other
duties.[Footnote 44] This prohibition has the practical effect of
precluding FTA from conducting benefit-cost analysis of proposed
transit projects.
Despite this prohibition, benefit-cost analysis could help FTA better
organize and evaluate information about proposed transit projects. Some
experts we spoke to said that benefit-cost analysis, in conjunction
with other qualitative evaluation measures, would be an ideal framework
for evaluating New Starts projects. Most experts we spoke to agreed
that, conceptually, benefit-cost analysis offers a full comparison of
transit projects' benefits and costs. One expert said that it is
appropriate to have an evaluation process that produces detailed
estimates of all benefits and costs so that projects with the highest
net benefits can be identified and funded because the New Starts'
program budget is limited. In the past, we have encouraged the use of
benefit-cost analysis in other areas, such as freight transportation,
and noted the usefulness of the analysis for federal transportation
decision makers.[Footnote 45] Some experts also maintained that most of
the information necessary for benefit-cost analysis is already produced
or available to project sponsors. Most experts we spoke to who
advocated using benefit-cost analysis, however, maintained that the
quantitative results of the analysis should be used in concert with
qualitative measures to account for those factors that cannot be
monetized. We have noted in the past that guidance on benefit-cost
analysis advises decision makers to augment the results of the analysis
with consideration of other factors, such as the equitable distribution
of benefits.[Footnote 46] Executive Order 12893 directs agencies to
assess benefits and costs of proposed infrastructure investments. In
addition, we and others, including the Office of Management and Budget
and DOT, have also identified benefit-cost analysis as a useful tool
for integrating the social, environmental, economic, and other effects
of investment alternatives and for helping transportation decision
makers identify projects with the greatest net benefits.[Footnote 47]
In this way, benefit-cost analysis could provide FTA with a systematic
and comprehensive assessment of proposed projects' impacts.
In addition to the legal prohibition on FTA monetizing certain project
benefits, there are many short-term challenges to implementing benefit-
cost analysis. First, according to some experts we spoke to and our
previous work, because local travel models produce outputs that become
inputs for benefit-cost analysis, this approach to evaluating projects
is limited by the previously mentioned limitations of local travel
models. Accordingly, some experts we spoke to maintained that the
results of benefit-cost analysis would not be reliable. FTA officials
also told us that many project sponsors do not have the technical
capacity to conduct benefit-cost analysis. A second challenge
identified by many experts and project sponsors is the difficulty of
monetizing certain project benefits and considering the distribution of
predicted benefits. For example, determining how to quantify and
monetize reductions in emissions and travel time can be challenging.
Although agency guidance exists, researchers do not always agree on the
appropriate methods for valuing these impacts. Additionally, while
benefit-cost analysis attempts to determine the net benefits of
projects, it does not usually consider the distribution of those
benefits across locations or populations or other equity concerns that
may exist. As two experts told us, and as we have noted in the past,
these distributional issues could be addressed within benefit-cost
analysis by, for example, weighting the benefits and costs to a
disadvantaged group differently than those to other segments of the
population. However, it can be difficult in practice to determine the
appropriate weights to assign to particular groups. Some experts and
project sponsors said that FTA should not adopt this approach to
evaluating projects because of these particular weaknesses.
An FTA official told us that they do not support using benefit-cost
analysis because of the challenges associated with monetizing benefits.
FTA officials also maintained that their current evaluation process
captures information similar to a formal benefit-cost analysis. They
also said that their current process is appropriate because the goal of
the New Starts evaluation process, given funding constraints, is to
produce a relative ranking of proposed projects, not to identify all
projects with positive net benefits. As we have previously stated,
FTA's emphasis on mobility improvements and reliance on certain proxy
measures in the current and proposed evaluation processes may
underestimate total project benefits, thereby impacting the relative
ranking of projects. In contrast, benefit-cost analysis would attempt
to monetize all benefits and costs, which experts told us would be a
more comprehensive approach to evaluating projects. Finally, an FTA
official we spoke with also noted that the statutory prohibition on
monetizing mobility improvements when evaluating projects prevents FTA
from using benefit-cost analysis for the New Starts program.
Evaluation Process Could Differ by Project Goal, but This Option May
Not Support National Priorities:
A third option to evaluate proposed transit projects is to evaluate
them differently based on their primary goal. Experts and projects
sponsors told us that transit projects have different and multiple
goals, from improving mobility to reducing greenhouse gas emissions.
(See figure 5 for examples of transit project goals.) Some experts and
project sponsors said that the New Starts program could focus more on
facilitating local transit goals, such as economic development, by
using different evaluation processes for projects with different goals.
They advocated for options that would emphasize local goals because
they said the practical effect of FTA's current evaluation process is
the exclusion of certain transit projects from funding consideration.
More specifically, projects with the goal of fostering high-density
development through the construction of transit stations often cannot
achieve a successful ranking under the New Starts process because they
generally are not predicted to create significant transit user
benefits. According to one expert we spoke to, this goal-focused option
could either involve different evaluation criteria for different types
of projects or consistent criteria but different weights for the
criteria based on the goal of the project. For example, projects with
the primary goal of catalyzing and managing local economic development
could be evaluated mainly on the basis of predicted economic
development effects and the extent of transit-supportive policies and
characteristics in the project corridor.
Figure 5: Examples of Potential Goals for Transit Projects:
[See PDF for image]
This figure is an illustration of examples of potential goals for
transit projects, as follows:
Transit project goals:
* Mobility improvements: As defined by FTA, reductions in the time or
cost of travel, as well as improvements in transportation reliability,
comfort, and convenience;
* Environmental benefits: Such as improved air quality, sustainable
building practices, and efficient energy use;
* Economic development/land use: Such as creation of high-density
development around transit stations or raising the property values in
an area.
Source: GAO analysis.
[End of figure]
Experts and project sponsors we spoke to said the main weakness of
using different evaluation frameworks is that federal transit spending
should reflect national priorities. More specifically, they said that
because the New Starts program is funded by the federal government,
projects should go through a national evaluation process designed to
support those projects that serve particular national goals. One expert
in particular said that FTA should retain its primary focus on funding
projects that improve mobility and not on those designed to change the
structure of cities.[Footnote 48] FTA officials also maintained that
projects should not be evaluated differently because the New Starts
program is a national program and, as such, should have an evaluation
process that reflects national priorities and is consistently applied
to all projects. Additionally, some experts we spoke to said that
establishing defensible and appropriate measures for different
evaluation processes could be difficult. Some experts also said that it
may be hard to separate projects into different categories, given the
fact that most projects have overlapping goals. Finally, some experts
expressed concern that project sponsors would self-select into the
evaluation process under which they score best. Such self-selection
could increase the total number of projects qualifying for New Starts
funding, while potentially decreasing the rigor of the selection
process. FTA officials also expressed this concern because potential
measures associated with certain goals, such as economic development,
are relatively subjective. The officials maintained that it would be
difficult to develop appropriate and defensible metrics to assess
projects with goals other than mobility improvements.
Evaluation Process Could Be Devolved to the States under Formula Grant
Program but Could Lack Federal Accountability:
According to some experts we spoke to, a fourth option is to eliminate
the evaluation process at the federal level and devolve this
responsibility to the states. In particular, these experts suggested
using a formula grant program to distribute New Starts funds, noting
that this option would result in projects that better reflect local
transit priorities. One expert we spoke to maintained that most transit
projects only have local or regional benefits and no national impacts,
and thus, should be controlled by states. A formula grant program in
particular, according to some of those experts, could ensure that local
areas build projects that meet their needs, as opposed to those that
meet FTA's expectations.
According to experts we spoke to, shifting the federal investment in
fixed guideway transit from a discretionary grant program to a formula
grant program would devolve the evaluation of projects to the state or
local levels. Formula grant programs allocate funds to states or their
subdivisions in accordance with a distribution formula prescribed in
law or regulation. Grant recipients may then allocate these funds to
specific projects based on program eligibility guidelines. One expert
we spoke to also suggested developing a large-scale transportation
formula grant program that would include money for New Starts projects.
Such a program could use performance-based indicators to make state
allocations.
Other experts we spoke to, however, said that establishing
accountability mechanisms for project performance under a formula
program could be difficult. Formula grant programs lodge decision
power, and thus accountability, at the state and local levels to
varying degrees and with varying constraints. The practical result of
this, as we have noted in our past work, is often that program-specific
performance information is collected through program operations, which
limits the ability of the federal government to hold grantees
accountable.[Footnote 49] Some formula grant programs' designs
inherently limit the prospect of collecting program-wide performance
data through program operations. As we have also previously reported,
many current surface transportation projects funded through formula
grant programs are not effective at addressing key transportation
challenges.[Footnote 50] They generally do not address these challenges
because the federal role is unclear and programs lack links to needs or
performance. Furthermore, devolving the evaluation process for proposed
transit projects would also eliminate the rigorous, national,
evaluation process FTA has developed--through the New Starts program--
which we have previously recognized as a model for other programs. More
specifically, we have noted that while the New Starts program requires
project sponsors to justify their proposed transit projects on the
basis of cost-effectiveness and other criteria, there are no similar
federal requirements for analyses of highway project benefits because
those projects are funded under a formula program.
Conclusions:
FTA's New Starts program is often cited as a model for other federal
transportation programs. FTA's recommendations for funding are based on
a rigorous examination of the benefits and costs of proposed projects,
and Congress has generally followed FTA's funding recommendations.
However, there is growing lack of confidence among Members of Congress
and the transit industry about the process and the results it produces.
For instance, FTA may be underestimating projects' benefits because
existing and proposed evaluation measures do not fully capture all
potential benefits, such as benefits to highway users and environmental
benefits. Capturing these other benefits potentially could change the
relative rankings of proposed projects and FTA's funding
recommendations. According to FTA officials and some experts we
interviewed, local models must be improved in order to develop and
employ better measures of project impacts. These models produce the
data necessary to measure potential benefits of transit projects, such
as the projects' impacts on highway congestion. However, due to
technical limitations, current models cannot be counted on to
accurately and reliably produce this information. Without improvements
to these models, FTA will have to continue using proxies for certain
benefits--which could lead to inaccurate assessments of projects'
benefits. Improving these models is a complex and costly endeavor--and
will likely require support from all levels of the government. However,
given that New Starts projects cost hundreds of millions of dollars, it
seems prudent that FTA and other federal, state, and local agencies
take steps to improve the models used to provide critical information
to policymakers about the merits of the projects and ultimately,
whether the projects should be implemented. Furthermore, the benefits
of improving local travel models would extend beyond transit projects,
as data from these models are used to inform regional transportation
planning for other modes, as well. The upcoming reauthorization of all
transportation programs, including the New Starts program, provides an
opportunity to seek additional resources to improve local travel
models.
FTA is working to improve the New Starts evaluation process and, in
particular, address the limitations associated with its current
measures. For example, FTA has issued a request for proposals to
develop approaches for predicting changes in highway user benefits,
which could help eliminate the need to use crude proxies in the
evaluation process and, therefore, more accurately measure project
benefits. However, FTA has not established a timeline for completing
this effort. Furthermore, FHWA has declined to participate in this
effort, even though the results could benefit all kinds of
transportation planning. In addition, although FTA has committed to
work with environmental experts to improve the environmental benefits
measures, FTA has not begun this effort, or established time frames for
initiating or completing this effort. Given that there is general
consensus that FTA's existing and proposed environmental benefits
measures do not meaningfully distinguish among projects, FTA should
work expeditiously to improve these measures before having project
sponsors develop and submit information that is not useful for
evaluation and rating purposes. In addition, FTA has worked to
incorporate qualitative information about certain project benefits in
the evaluation process, which can help ensure that all project benefits
are fully considered. However, the inclusion of qualitative information
in the evaluation process does not negate the need for FTA to work to
improve existing or develop new quantitative measures for the different
evaluation criteria.
There are a number of alternatives FTA can consider as it explores
options for revamping the New Starts program. The NRPM presents one way
to modify the existing evaluation framework, but there are also several
different options that could serve as a means to determine which
transit projects should receive New Starts funding. In particular, our
past work and some of the experts we spoke to identified benefit-cost
analysis as a viable tool that could provide a comprehensive analysis
of projects' costs and benefits over time. However, FTA's ability to
consider this approach is constrained by the current prohibition on
placing dollar values on mobility improvements. Going forward, it is
important that FTA have the flexibility to consider a wide range of
approaches for evaluating transit projects, including benefit-cost
analysis, as it seeks to improve the New Starts program.
Recommendations for Executive Action:
To improve the New Starts evaluation process and the measures of
project benefits, which could change the relative ranking of projects,
we recommend that the Secretary of Transportation take the following
five actions:
(1) Seek additional resources to improve local travel models in the
next authorizing legislation;
(2) Seek a legislative change to allow FTA to consider the dollar value
of mobility improvements in evaluating projects, developing
regulations, or carrying out any other duties;
(3) Direct the Administrator of FTA to establish a timeline for
issuing, awarding, and implementing the result of its request for
proposals on short-and long-term approaches to measuring highway user
benefits from transit improvements;
(4) Direct the Administrator of FTA to establish a timeline for
initiating and completing its longer-term effort to develop more robust
measures of transit projects' environmental benefits that are
practically useful in distinguishing among proposed projects, including
consultation with the transit community, and;
(5) Direct the Administrators of FTA and FHWA to collaborate in efforts
to improve the consistency and reliability of local travel models,
including the aforementioned request for proposals on approaches to
measuring highway user benefits.
Agency Comments and Our Evaluation:
We provided a draft of this report to DOT for review and comment. DOT
generally agreed with the findings and recommendations in this report,
and provided clarifying comments and technical corrections, which we
incorporated, as appropriate.
We are sending copies of this report to DOT and appropriate
congressional committees. We will also make copies available to others
upon request. In addition, the report will be available at no charge on
the GAO Web site at [hyperlink, http://www.gao.gov]. If you have any
questions about this report, please contact me at siggerudk@gao.gov or
(202) 512-2834. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. Key contributors to this report are listed in appendix IV.
Signed by:
Katherine Siggerud:
Managing Director, Physical Infrastructure Issues:
[End of section]
Appendix I: Summary of New Starts and Small Starts Projects Evaluated,
Rated, and Recommended for Funding for FY 2009:
Administration Requests $1.62 Billion for New Starts and Small Starts
Projects for Fiscal Year 2009:
The Federal Transit Administration (FTA) evaluated and rated 29 New
Starts, Small Starts, and Very Small Starts projects for funding during
the fiscal year 2009 evaluation cycle. FTA evaluated and rated 13 New
Starts projects, 2 of which had pending full funding grant agreements
(FFGA) and were recommended for funding. FTA did not recommend any new
New Starts projects for funding this year. FTA also evaluated and rated
16 Small Starts and Very Small Starts projects and recommended 13 of
these projects for funding. The fiscal year 2009 President's budget
requests $1.62 billion in New Starts funding, the majority of which is
for 15 projects with existing FFGAs.
FTA Evaluated and Rated 13 New Starts Projects but Did Not Recommend
Any New Projects for Funding:
FTA identified 16 New Starts projects during the fiscal year 2009
cycle, including 2 projects with pending FFGAs and 14 projects in
preliminary engineering and final design. (See table 4 for a full list
of these projects.) Of the 16 total projects, 13 projects were
evaluated and rated using the newly instituted five-level scale, and 3
projects were statutorily exempt from being rated.[Footnote 51]
Table 4: Pending FFGAs and Projects in Final Design and Preliminary
Engineering:
Project name: Pending FFGAs[A]: West Corridor Light Rail Transit (LRT);
Location: Denver, Colo.;
Total capital cost (dollars in millions): $656.8;
Federal share of total capital costs (percent): 44%;
Overall project rating: Medium-high.
Project name: Pending FFGAs[A]: University Link LRT Extension;
Location: Seattle, Wash.;
Total capital cost (dollars in millions): $1,798.1;
Federal share of total capital costs (percent): 42;
Overall project rating: High.
Project name: Final design: Hartford Busway;
Location: Hartford, Conn.;
Total capital cost (dollars in millions): $458.8;
Federal share of total capital costs (percent): 60;
Overall project rating: Medium.
Project name: Final design: Urban Transitway Phase II;
Location: Stamford, Conn.;
Total capital cost (dollars in millions): $48.3;
Federal share of total capital costs (percent): 51;
Overall project rating: Exempt.
Project name: Final design: Wilmington to Newark Commuter Rail
Improvements;
Location: Wilmington, Del.;
Total capital cost (dollars in millions): $78.4;
Federal share of total capital costs (percent): 32;
Overall project rating: Exempt.
Project name: Final design: South County Commuter Rail;
Location: Providence, R.I.;
Total capital cost (dollars in millions): $49.2;
Federal share of total capital costs (percent): 51;
Overall project rating: Exempt.
Project name: Preliminary engineering: South Sacramento Corridor Phase
2;
Location: Sacramento, Calif.;
Total capital cost (dollars in millions): $226.2;
Federal share of total capital costs (percent): 50;
Overall project rating: Medium-high.
Project name: Preliminary engineering: Central Subway LRT;
Location: San Francisco, Calif.;
Total capital cost (dollars in millions): $1,289.8;
Federal share of total capital costs (percent): 59;
Overall project rating: Medium-high.
Project name: Preliminary engineering: Orange Line Phase 2: North
Corridor Metrorail Extension;
Location: Miami, Fla.;
Total capital cost (dollars in millions): $1,605.4;
Federal share of total capital costs (percent): 44;
Overall project rating: Medium-low.
Project name: Preliminary engineering: Central Florida Commuter Rail
Transit - Initial Operating Segment;
Location: Orlando, Fla.;
Total capital cost (dollars in millions): $416.7;
Federal share of total capital costs (percent): 50;
Overall project rating: Medium-high.
Project name: Preliminary engineering: Silver Line Phase III;
Location: Boston, Mass.;
Total capital cost (dollars in millions): $1,167.3;
Federal share of total capital costs (percent): 60;
Overall project rating: Medium.
Project name: Preliminary engineering: Central Corridor LRT;
Location: St. Paul, Minn.;
Total capital cost (dollars in millions): $932.3;
Federal share of total capital costs (percent): 50;
Overall project rating: Medium.
Project name: Preliminary engineering: Northeast Corridor Light Rail
Project;
Location: Charlotte, N.C.;
Total capital cost (dollars in millions): $749;
Federal share of total capital costs (percent): 50;
Overall project rating: Medium-high.
Project name: Preliminary engineering: Access to the Region's Core;
Location: Northern New Jersey;
Total capital cost (dollars in millions): $7,263.5;
Federal share of total capital costs (percent): 41;
Overall project rating: Medium-high.
Project name: Preliminary engineering: Mid-Jordan LRT;
Location: Salt Lake City, Utah;
Total capital cost (dollars in millions): $553.7;
Federal share of total capital costs (percent): 78;
Overall project rating: Medium-high.
Project name: Preliminary engineering: Dulles Corridor Metrorail
Project - Extension to Wiehle Avenue[B];
Location: Northern Virginia;
Total capital cost (dollars in millions): $2,960.8;
Federal share of total capital costs (percent): 30;
Overall project rating: Medium.
Source: GAO summary of New Starts fiscal year 2009 Annual Report.
[A] Pending FFGAs refer to projects that FTA expects will execute an
FFGA within the upcoming fiscal year. According to FTA, all projects
seeking a funding recommendation, including pending FFGAs, are
evaluated and rated during the evaluation cycle. Both Seattle and
Denver were evaluated and rated because they were seeking
recommendations for an FFGA in the fiscal year 2009 report.
[B] The Dulles Corridor Metrorail Project was not rated in FTA's fiscal
year 2009 Annual Report on New Starts projects that was released in
February 2008. However, following FTA's review of additional
documentation related to the project's costs, financial plan, and
management processes, the project was evaluated and received its final
overall rating in May 2008.
[End of table]
Although they evaluated and rated fewer New Starts projects during the
fiscal year 2009 cycle than in previous years, FTA officials told us
that this decrease does not indicate that there are fewer projects in
the pipeline. They stated that the Annual Report only provides a
snapshot of the total portfolio of projects in development or under
construction. As a result, projects that have existing FFGAs or those
that are currently in alternatives analysis are not included in this
list. Since last year's New Starts evaluation and rating cycle, four
projects in the pipeline "graduated" from final design and received
FFGAs, and one sponsor withdrew two projects from the process after
changing the project type in both corridors from bus rapid transit to
light rail rapid transit. FTA expects that the revised projects will
return to the pipeline and progress toward an FFGA in the future. FTA
officials also anticipate that several other projects that are
currently in alternatives analysis will move into preliminary
engineering at some point in the near future, at which point they will
be evaluated and rated.
FTA did not recommend any new projects for funding in the current
evaluation cycle but did recommend funding for two projects with
pending FFGAs: the West Corridor Light Rail Transit (LRT) in Denver and
the University Link LRT Extension in Seattle. In its Annual Report, FTA
states that both of these projects meet the New Starts criteria, are at
an advanced stage of development with few remaining uncertainties, and
are expected to be ready for an FFGA prior to or during fiscal year
2009. The total capital cost of these two projects is estimated to be
$2.46 billion, with the total federal New Starts share for the West
Corridor LRT at 44 percent and the University Link LRT extension at 42
percent of the total cost, respectively. FTA also recommended reserving
$78 million[Footnote 52] in New Starts funding for final design
activities for projects that will reach final design prior to the
development of the fiscal year 2009 appropriations bill.[Footnote 53]
Unlike in previous years, FTA has not specified which projects will be
eligible for this funding or allocated a particular amount for any
given project. According to the Annual Report and officials we spoke to
at FTA, this approach will allow the agency to make "real time" funding
recommendations as project uncertainties are mitigated and Congress
makes final appropriations decisions. FTA does not expect that all of
the projects in preliminary engineering will advance to final design in
fiscal year 2009 (see table 4).
FTA Evaluated and Rated 16 Small Starts and Very Small Starts Projects
and Recommended Funding for 13 Projects:
FTA evaluated and rated 16 eligible Small Starts and Very Small Starts
projects, including 12 projects that were advanced into project
development during this cycle and 4 existing Small Starts projects that
were not fully funded in fiscal year 2008.[Footnote 54] Ten projects
received a "medium" rating and 6 projects received a "medium-high"
rating. FTA recommended 13 of these 16 projects for funding.[Footnote
55] (See table 5 for a list of FTA's funding recommendations for fiscal
year 2009.) The total capital cost of the 13 projects that FTA
recommended for funding is estimated to be $771.6 million, and the
total Small Starts, including Very Small Starts, share is expected be
about $451.6 million. Most of these projects are proposed to be funded
under a multiyear Project Construction Grant Agreement. However, three
projects, which have requested less than $25 million in total Small
Starts funding, are proposed in this budget to be funded under one-year
capital grants.
Table 5: Fiscal Year 2009 Small Starts and Very Small Starts Funding
Recommendations:
Project name: Mountain Links Bus Rapid Transit (BRT);
Location: Flagstaff, Ariz.;
Total capital cost (dollars in millions): $10.4;
Federal share of capital cost (percent): 60%;
Type of project: Very Small Starts.
Project name: Livermore-Amador Route 10 BRT;
Location: Livermore, Calif.;
Total capital cost (dollars in millions): $21.7;
Federal share of capital cost (percent): 51;
Type of project: Very Small Starts.
Project name: Metro Rapid Bus System Gap Closure;
Location: Los Angeles, Calif.;
Total capital cost (dollars in millions): $25.7;
Federal share of capital cost (percent): 65;
Type of project: Very Small Starts.
Project name: Wilshire Boulevard Bus-Only Lane;
Location: Los Angeles, Calif.;
Total capital cost (dollars in millions): $31.5;
Federal share of capital cost (percent): 74;
Type of project: Very Small Starts.
Project name: Perris Valley Line;
Location: Riverside, Calif.;
Total capital cost (dollars in millions): $168.3;
Federal share of capital cost (percent): 45;
Type of project: Small Starts.
Project name: Mid-City Rapid; [Empty];
Location: San Diego, Calif.;
Total capital cost (dollars in millions): $43.3;
Federal share of capital cost (percent): 50;
Type of project: Very Small Starts.
Project name: Mason Corridor BRT;
Location: Fort Collins, Colo.;
Total capital cost (dollars in millions): $74.2;
Federal share of capital cost (percent): 80;
Type of project: Small Starts.
Project name: Commuter Rail Improvements;
Location: Fitchburg, Mass.;
Total capital cost (dollars in millions): $150;
Federal share of capital cost (percent): 50;
Type of project: Small Starts.
Project name: Troost Corridor BRT;
Location: Kansas City, Mo.;
Total capital cost (dollars in millions): $30.7;
Federal share of capital cost (percent): 80;
Type of project: Very Small Starts.
Project name: Streetcar Loop;
Location: Portland, Ore.;
Total capital cost (dollars in millions): $126.9;
Federal share of capital cost (percent): 59;
Type of project: Small Starts.
Project name: Pioneer Parkway EmX BRT;
Location: Springfield, Ore.;
Total capital cost (dollars in millions): $37.0;
Federal share of capital cost (percent): 80;
Type of project: Very Small Starts.
Project name: Bellevue-Redmond BRT;
Location: King County, Wash.;
Total capital cost (dollars in millions): $27.0;
Federal share of capital cost (percent): 75;
Type of project: Very Small Starts.
Project name: Pacific Highway South BRT;
Location: King County, Wash.;
Total capital cost (dollars in millions): $25.1;
Federal share of capital cost (percent): 56;
Type of project: Very Small Starts.
Source: GAO summary of information in the New Starts fiscal year 2009
Annual Report.
[End of table]
Administration's Fiscal Year 2009 Budget Recommends $1.62 Billion for
the New Starts Program:
The administration's fiscal year 2009 budget proposal recommends that
$1.62 billion be made available for the New Starts program. This amount
is $51.7 million more than the program's fiscal year 2008
appropriation. Figure 6 illustrates the planned uses of the
administration's proposed request for the New Starts fiscal year 2009
budget, including the following:
* $1,146.62 million would be allocated among the 15 projects with
existing FFGAs;
* $160 million would be allocated among 2 projects with pending FFGAs;
* $78 million would be allocated to projects that will reach final
design before the end of this fiscal year;
* $200 million would be allocated for Small Starts projects;
* $20 million for ferry capital projects (Alaska and Hawaii) and Denali
Commission; and:
* $16.2 million for oversight activities.
Figure 6: Allocation of Administration's Proposed Fiscal Year 2009
Budget for New Starts:
[See PDF for image]
This figure is a pie-chart depicting the following data:
Allocation of Administration's Proposed Fiscal Year 2009 Budget for New
Starts:
Existing FFGAs: 70% ($1,146.62 million);
Small Starts projects: 12% ($200 million);
Pending FFGAs: 10% ($160 million );
Final design activities: 5% ($78 million);
Ferry Capital Projects (Alaska and Hawaii) and Denali Commission: 1%
($20 million);
Oversight activities: 1% ($16.2 million).
Source: GAO analysis of FTA data.
Notes: FTA is authorized to use up to 1 percent of amounts made
available for the New Starts program for project management oversight
activities.
Federal statute requires that specified amounts of New Starts funds be
set aside annually for projects in Alaska and Hawaii, for fixed
guideway ferry systems and extension projects utilizing ferry boats,
ferry boat terminals, or approaches to ferry boat terminals.
FTA is also authorized to provide $5 million for each fiscal year from
2006 to 2009 for the Denali Commission, which provides critical
utilities, infrastructure, and economic support throughout Alaska,
particularly in remote communities.
Percentages do not add to 100 percent due to rounding.
[End of figure]
[End of section]
Appendix II: Scope and Methodology:
To address our objectives, we reviewed previous GAO reports, FTA's
existing and proposed New Starts policy guidance, FTA's August 2007
Notice of Proposed Rulemaking (NPRM) for New Starts, and the provisions
of SAFETEA-LU that address the New Starts program to identify the
information captured by the current and proposed New Starts project
justification criteria. We also reviewed various pieces of legislation,
including SAFETEA-LU and New Starts authorizing legislation, along with
legislative history, to determine the extent to which New Starts
program goals have been expressed or defined in law. Furthermore, we
reviewed FTA's Annual Report on New Starts for fiscal year 2009 to
determine the number of projects evaluated, rated, and recommended for
funding, the amount of funding requested for these projects, and the
total costs of proposed projects.
We also examined a sample of public comments submitted in response to
the proposed revisions to FTA's current evaluation process, as
described in the NPRM.[Footnote 56] First, we reviewed all 104 comments
submitted to the docket to understand the range of perspectives on the
proposed revisions described in the NPRM. Second, following this
review, we conducted a more in-depth review of 13 comments submitted by
(1) project sponsors we interviewed; (2) professional and advocacy
groups we interviewed; and (3) organizations submitting extensive and
relevant comments, as determined by team members. Third, upon
completion of this analysis, we also reviewed 27 of the remaining 91
comments. After sorting the remaining comments, we randomly selected
comments in proportion to the total number of comments received by (1)
geographic diversity; (2) relevance of comment to FTA's proposals; and
(3) diversity of opinion. We categorized and analyzed comments to
determine the frequency of particular perspectives and opinions about
FTA's proposed revisions, as well as other options for evaluating
projects. Because the comments were selected as a nonprobability
sample, the results cannot be generalized to all comments.
We interviewed FTA and transit industry officials to get an in-depth
assessment of the information captured by the current and proposed New
Starts project justification measures as well as how FTA's current
evaluation process influences projects' cost, schedule, and design. We
also interviewed FTA officials to discuss how the design and use of
these measures impacts the calculation of project benefits, how the
proposed revisions respond to SAFETEA-LU and past concerns voiced by
the transit industry, and what other options they have considered to
measure different project justification criteria. To learn more about
the ongoing rulemaking process, we also attended New Starts Listening
Sessions in Washington, D.C., and Charlotte, North Carolina, in October
2007. We also attended FTA's expert panel discussion to identify
approaches for incorporating land use and economic development into the
New Starts evaluation framework. In addition, we interviewed three
industry associations (that represent project sponsors) that
participate closely in these programs: the American Public
Transportation Association, New Starts Working Group, and Reconnecting
America.
We also interviewed 11 project sponsors, including both Small Starts
projects in the project development phase and New Starts projects in
the preliminary engineering or final design stages for the fiscal year
2009 evaluation cycle. We conducted semistructured interviews with the
project sponsors to gather additional information on FTA's current
evaluation process; how FTA's evaluation measures influence projects'
cost, schedule, and design; and other options for evaluating proposed
transit projects. We selected these projects based on the following
criteria: (1) projects seeking different types of funding (e.g., New
Starts or Small Starts); (2) projects involving different modes of
transit (e.g., rail, light rail, or bus); (3) projects in different
stages of project development (e.g., preliminary engineering or final
design); (4) projects of different sizes (based on the total capital
cost and ridership projections); and (5) projects from different
geographic areas. Because the 11 projects were selected as a
nonprobability sample, the results cannot be generalized to all
projects. Table 6 lists the New Starts and Small Starts project
sponsors we interviewed for our review.
Table 6: New Starts and Small Starts Project Sponsors Interviewed:
Name of project sponsor: Charlotte Area Transit System;
Location: Charlotte, N.C.;
Project type: New Starts.
Name of project sponsor: Massachusetts Bay Transit Authority;
Location: Boston, Mass.;
Project type: New Starts.
Name of project sponsor: Metropolitan Miami-Dade County Transit
Authority;
Location: Miami, Fla.;
Project type: New Starts.
Name of project sponsor: New Jersey Transit;
Location: Northern New Jersey;
Project type: New Starts.
Name of project sponsor: Sound Transit;
Location: Seattle, Wash.;
Project type: New Starts.
Name of project sponsor: Metropolitan Washington Airports Authority;
Location: Northern Virginia;
Project type: New Starts.
Name of project sponsor: City of Portland;
Location: Portland, Ore.;
Project type: Small Starts.
Name of project sponsor: City of Stamford;
Location: Stamford, Conn.;
Project type: Small Starts.
Name of project sponsor: Northern Arizona Intergovernmental Public
Transportation Authority;
Location: Flagstaff, Ariz.;
Project type: Small Starts.
Name of project sponsor: Riverside County Transportation Commission;
Location: Riverside, Calif.;
Project type: Small Starts.
Name of project sponsor: Sacramento Regional Transit District;
Location: Sacramento, Calif.;
Project type: Small Starts.
Source: GAO.
[End of table]
To further address our objectives, we interviewed a variety of
transportation experts and consultants to obtain their perspectives on
FTA's current evaluation process and other options for evaluating
proposed transit projects. We used a semistructured interview guide and
followed up by e-mail to collect comparable information from all
experts. We selected an initial group of transportation experts to
interview based on their past participation in GAO and FTA expert
panels on similar topics and their research on transit issues,
including the New Starts program.[Footnote 57] During these initial
interviews, we solicited recommendations of other experts we should
interview. Using this snowballing technique, we selected the most
frequently recommended experts for interviews, as well as those with
the most relevant expertise. Table 7 lists the experts we interviewed.
Table 7: Experts Interviewed for Fiscal Year 2009 New Starts Review:
Name: Chandra Bhat;
Title: Adnan Abou-Ayyash Centennial Professor in Transportation
Engineering;
Affiliation: University of Texas at Austin, Department of Civil,
Architectural and Environmental Engineering.
Name: Robert Cervero;
Title: Professor of City and Regional Planning;
Affiliation: University of California, Berkeley.
Name: Elizabeth Deakin;
Title: Professor of City and Regional Planning/Director of the
University of California Transportation Center;
Affiliation: University of California, Berkeley.
Name: Genevieve Giuliano;
Title: Professor and Senior Associate Dean for Research and Technology,
School of Policy, Planning, and Development;
Affiliation: University of Southern California.
Name: José A. Gómez Ibanéz;
Title: Professor of Urban Planning and Public Policy;
Affiliation: Harvard University.
Name: Ronald Kirby;
Title: Director of Transportation Planning;
Affiliation: Metropolitan Washington Council of Governments.
Name: Kara Kockelman;
Title: Associate Professor and William J. Murray Jr. Fellow;
Affiliation: University of Texas at Austin, Department of Civil,
Architectural and Environmental Engineering.
Name: David Lewis;
Title: Chief Economist;
Affiliation: HDR Inc.
Name: Eric Miller;
Title: Bahen-Tanenbaum Professor of Transportation Engineering and
Planning;
Affiliation: University of Toronto.
Name: Don Pickrell;
Title: Chief Economist;
Affiliation: Volpe Center, Research and Innovative Technology
Administration, U.S. Department of Transportation.
Name: John Pucher;
Title: Professor of Urban Planning, Research Associate in the Alan M.
Voorhees Transportation Center;
Affiliation: Rutgers University.
Name: Michael Roschlau;
Title: President and Chief Executive Officer;
Affiliation: Canadian Urban Transit Association.
Name: Frederick Salvucci;
Title: Senior Lecturer and Senior Research Associate;
Affiliation: Center for Transportation and Logistics, Massachusetts
Institute of Technology.
Name: Martin Wachs;
Title: Director, Transportation, Space, and Technology Program;
Affiliation: RAND Corporation.
Name: Nigel Wilson;
Title: Professor of Civil and Environmental Engineering;
Affiliation: Massachusetts Institute of Technology.
Source: GAO.
[End of table]
Following the interviews, team members categorized and analyzed the
experts' comments to determine the frequency of particular perspectives
about FTA's current evaluation process and other options for evaluating
projects. To supplement the perspectives of these experts, we also
interviewed other scholars and consultants with specific knowledge of
the New Starts project evaluation process, including Don Emerson,
Principal Consultant, Parsons Brinckerhoff Consulting; Laurie Hussey,
Consultant, Cambridge Systematics, Inc.; Terry Moore, Planning
Director, Land-Use and Transportation Planning, ECONorthwest; Kenneth
Orski, Editor and Publisher, Innovation Briefs; Randy Pozdena, Senior
Economist, Monetary Policy and Industrial Organization, ECONorthwest;
Michael Replogle, Transportation Director, Environmental Defense; and
Ronald Utt, Herbert and Joyce Morgan Senior Research Fellow, Heritage
Foundation.
We also reviewed academic and professional literature about the impact
of public transit on mobility, economic development, and the
environment. The purpose of our literature review was to assess the
accuracy of particular assertions made by experts, project sponsors,
and government officials we interviewed. Our literature review included
articles identified through searches of research databases and the
Internet, as well as suggestions of experts we interviewed. Team
members analyzed and summarized the evidence from these articles in
consultation with a GAO methodologist and economist.
We conducted this performance audit from October 2007 to June 2008 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
[End of section]
Appendix III: Explanation of FTA's Calculation of Transportation System
User Benefits:
The transportation system user benefits (TSUB) measure is intended to
capture all the significant user benefits of a proposed transit
project. The measure includes predicted travel time savings and
accounts for other benefits by quantifying the effect of nontravel time
factors that influence travel behavior. The unit of the TSUB measure is
equivalent to minutes of in-vehicle travel time.
Project sponsors use local travel demand models to forecast ridership
and simulate trips taken in 2030, which is the forecast year used for
estimating benefits over time, for two alternatives. The baseline
alternative assumes low-cost improvements to the transportation
network, while the second alternative (the "build alternative") assumes
the proposed New Starts transit project (e.g., fixed guideway transit
infrastructure investment) is constructed.
Travel time savings from a proposed transit project can result from a
shorter wait, a shorter walk, or shorter in-vehicle times. To
adequately account for the time saved for each of these, the predicted
travel time savings for wait and walk times are weighted by a factor of
two or three, compared to in-vehicle time savings, because behavioral
surveys have shown that travelers perceive these out-of-vehicle times
as more onerous. The exact weighting factor is usually derived from
local travel models calibrated based on local travel surveys.
Other factors beyond travel time--namely, travel time reliability and
the convenience and comfort of the travel mode--are also incorporated
into the measure of user benefits through what is commonly referred to
as a modal constant. The modal constant varies by locality based on the
results of the model's calibration. Local models are generally
calibrated by adjusting the modal constant until the model accurately
predicts current travel patterns. Once a model is calibrated with a
particular constant, it is used to forecast future travel times, and
thus travel time savings, for the baseline and build alternatives.
These travel time savings, reflecting both actual time savings and
nontravel time factors, are referred to as user benefits.
The TSUB measure values user benefits differently for different
individuals. More specifically, it values the benefits of predicted
users of the project differently based on the travel mode they are
switching from (e.g., automobile or transit). Behavioral surveys have
shown that automobile users react differently to the user benefits
created by a transit project. Some require very small reductions in
transit travel time to change their travel mode from automobile to
transit (i.e., the build alternative) because they are relatively
indifferent between the existing transit option and automobile travel.
These travelers receive benefits, which economists call gains in
consumer surplus, because the reduction in transit travel times is
greater than what is required to induce their change in travel mode.
[Footnote 58] Others require the transit project's full measure of time
savings before they perceive any advantage to transit and change their
mode. These travelers, even though they choose to switch modes, receive
little gain in consumer surplus. In between these two kinds of
travelers are those with a range of preferences. Accordingly, the
"average" traveler that changes to the proposed transit project from
automobile travel requires half of the time savings created by the
project to change, and thus receives half of the project's benefits as
a gain in consumer surplus. For example, if a transit project is
introduced that makes travel in a particular corridor 10 minutes faster
than driving an automobile, the average benefit to an automobile user
switching to transit will be 5 minutes because some will require time
savings of less than 5 minutes to change modes and some will require
more. To account for this variation, FTA divides the total predicted
time savings for new transit riders by two when calculating user
benefits because, on average, only half of the benefits are received by
those travelers as gains in consumer surplus while the other half of
the benefits are needed to induce the change in mode and do not
represent a net benefit gain. Alternatively, individuals who switch
transit modes--from bus in the baseline alternative to a new light
rail, for example--would get the full 10 minute benefit of the switch
because no benefit is needed to induce a mode shift since they are
already transit users. These transit users take advantage of the full
travel time savings.
Transit projects can also create benefits for those who do not choose
to use them. For example, a transit project that reduces the number of
automobile travelers may reduce overall highway congestion. FTA does
not currently credit proposed projects with predicted benefits to
highway users because (1) FTA has found that most travel models around
the country do not predict plausible changes in highway speeds
resulting from transit improvements and (2) the absence of a consistent
method for highway speed prediction leads directly to potentially large
differences in the predicted benefits of transit projects with similar
impacts. To account for benefits to highway users, such as reduced
congestion as the result of more transit users, FTA raises the
breakpoints for the cost-effectiveness criterion by 20 percent, since
they are only using the transit user benefits as the denominator of
cost-effectiveness.
After accounting for factors that influence travel behavior as noted
above, travel times are compared between the baseline alternative and
build alternatives to produce the estimate of user benefits. That
measure of user benefits, TSUB, becomes the denominator in the
calculation of FTA's cost-effectiveness criterion.
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
Katherine Siggerud, (202) 512-2834 or siggerudk@gao.gov:
Staff Acknowledgments:
In addition to the individual named above, Nikki Clowers, Assistant
Director; Vidhya Ananthakrishnan; Kyle Browning; Lauren Calhoun; Jay
Cherlow; David Hooper; Delwen Jones; Sara Ann Moessbauer; Josh Ormond;
and Susan Zimmerman made key contributions to this report.
[End of section]
Footnotes:
[1] Fixed guideway systems use and occupy a separate right-of-way for
the exclusive use of public transportation services. These fixed
guideway systems include fixed rail, exclusive lanes for buses and
other high-occupancy vehicles, and other systems. An FFGA establishes
the terms and conditions for federal funds available for the project,
including the maximum amount of federal funds available.
[2] For more information on changes SAFETEA-LU made to the New Starts
program and the status of their implementation, see GAO, Public
Transportation: New Starts Program in a Period of Transition,
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-819] (Washington,
D.C.: Aug. 30, 2005) and GAO, Public Transportation: Future Demand Is
Likely for New Starts and Small Starts Programs, but Improvements
Needed to the Small Starts Application Process, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-917] (Washington, D.C.: July
27, 2007).
[3] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-917].
[4] 72 Fed. Reg. 43328 (Aug. 3, 2007).
[5] Pub. L. No. 110-161, Division K, Title I, Sec. 170, 121 Stat. 2401,
Dec. 26, 2007. "None of the funds provided or limited under this Act
may be used to issue a final regulation under section 5309 of title 49
[i.e., New Starts], United States Code, except that the Federal Transit
Administration may continue to review comments received on the proposed
rule (Docket No. FTA-2006-25737)."
[6] According to FTA officials, the TSUB measure and existing software
are designed to capture benefits that accrue to highway users, but the
forecasts used by local models are not reliable and as a result, are
not used.
[7] The Very Small Starts program is a project eligibility category
introduced by FTA in 2006 for projects with a total capital cost of
less than $50 million.
[8] 49 U.S.C. § 5309. Pub. L. No. 105-178, Section 3010, 112 Stat. 357,
June 9, 1998. The legislative record is silent as to why this provision
was enacted.
[9] During the preliminary engineering phase, project sponsors refine
the design of the proposal, taking into consideration all reasonable
design alternatives and estimating each alternative's costs, benefits,
and impacts (e.g., financial or environmental). According to FTA
officials, to gain approval for entry into preliminary engineering, a
project must (1) be identified through the alternatives analysis
process, (2) be included in the region's long-term transportation plan,
(3) meet the statutorily defined project justification and financial
criteria, and (4) demonstrate that the sponsors have the technical
capability to manage the project during the preliminary engineering
phase. Some federal New Starts funding is available to projects for
preliminary engineering activities, if so appropriated by Congress.
[10] Final design is the last phase of project development before
construction and may include right-of-way acquisition, utility
relocation, and the preparation of final construction plans and cost
estimates.
[11] The exceptions to the evaluation process are statutorily "exempt"
projects, which are those with requests for less than $25 million in
New Starts funding. Sponsors of these projects are not required to
submit project justification information (although FTA encourages the
sponsors to do so). FTA does not rate these projects. As a result, the
number of projects in the preliminary engineering or final design
phases may be greater than the number of projects evaluated and rated
by FTA.
[12] The administration's funding recommendations are made in the
President's budget and are included in FTA's annual New Starts report
to Congress, which is released each February in conjunction with the
President's budget.
[13] For more information on the changes SAFETEA-LU made to the New
Starts program and the status of their implementation, see [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-06-819] and [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-917].
[14] The legislation also requires that projects be funded only if they
are justified based on a comprehensive review of its (1) mobility
improvements, (2) environmental benefits, (3) cost-effectiveness, (4)
operating efficiencies, (5) economic development effects, and (6)
public transportation supportive land use policies and future patterns.
The legislation also lists a number of factors to be analyzed,
evaluated, and considered, including congestion relief, improved
mobility, air and noise pollution, and energy consumption.
[15] Small Starts projects are defined as those that are requesting
less than $75 million in federal funding and have a total estimated net
capital cost of less than $250 million. According to FTA's guidance,
Small Starts projects must (a) meet the definition of a fixed guideway
for at least 50 percent of the project length in the peak period or (b)
be a corridor-based bus project with the following minimum elements:
substantial transit stations; traffic signal priority/pre-emption, to
the extent, if any, that there are traffic signals on the corridor; low-
floor vehicles or level boarding; branding of the proposed service; and
10 minute peak/15 minute off-peak running times (i.e., headways) or
better while operating at least 14 hours per weekday.
[16] Very Small Starts projects must meet the same eligibility
requirements as Small Starts projects and be located in corridors with
more than 3,000 existing riders per average weekday who will benefit
from the proposed project. In addition, the projects must have a total
capital cost of less than $50 million (for all project elements) and a
per-mile cost of less than $3 million, excluding rolling stock (e.g.,
train cars).
[17] FTA is revising its evaluation and ratings process to comply with
SAFETEA-LU through the rulemaking process previously discussed.
However, as previously stated, Congress prohibited FTA from issuing the
final rule this fiscal year.
[18] Under federal planning requirements, states and metropolitan
planning organizations (MPOs) are required to establish a process for
collecting and analyzing data to evaluate different transportation
alternatives and use the resulting information to establish priorities
for improving local assets. As part of this process, planners may
develop local travel models and performance measures to evaluate
existing or proposed projects. Local travel models estimate future
travel demand and analyze the impacts of alternative transportation
investment scenarios.
[19] See appendix III for a more detailed description of how TSUB is
determined.
[20] FTA uses the following breakpoints to assign projects a cost-
effectiveness rating: $11.99 and under are rated high; $12.00 to $15.49
are rated medium-high; $15.50 to $23.99 are rated medium; $24.00 to
$29.99 are rated medium-low; and $30.00 and over are rated low. These
breakpoints are adjusted annually for inflation.
[21] TRB, Transit Cooperative Research Program Report 78, Estimating
the Benefits and Costs of Public Transit Projects: A Guidebook for
Practitioners (Washington, D.C., 2002).
[22] FTA considers the current air quality designation by the
Environmental Protection Agency (EPA) for the metropolitan region in
which the proposed project is located, indicating the severity of the
metropolitan area's noncompliance with the health-based EPA standard
for the pollutant or its compliance with that standard as the current
measure of environmental benefits.
[23] Specifically, FTA's July 2007 guidance notes that when mobility
improvements are rated Low, the summary rating will "round down" to the
lower of the two ratings; for all other mobility improvement ratings
(and for all Small Starts projects, which are not rated for mobility
improvements), the rating is "rounded-up" to establish the summary
project justification rating. For example, a New Starts project with a
cost-effectiveness rating of medium-high and a land use rating of low-
-along with a mobility improvements rating of medium--would receive a
summary project justification rating of medium.
[24] GAO, Opportunities Exist to Improve the Communication and
Transparency of Changes Made to the New Starts Program, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-05-674] (Washington, D.C.: June
28, 2005).
[25] Option value refers to the benefit that some transit users receive
by having transit service as an option for the future or in certain
circumstances. Agglomeration effects arise when the clustering of
business activity creates economies of scale or if infrastructure cost
savings result from compact development, both of which can be indirect
results of transit investments.
[26] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-917].
[27] See GAO, Highway and Transit Investments: Options for Improving
Information on Projects' Benefits and Costs and Increasing
Accountability for Results, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-05-172] (Washington, D.C.: Jan. 24, 2005).
[28] FTA requires agencies to hold land use and travel patterns
constant when comparing user benefits under the baseline alternative
(which assumes low-cost improvements are made to the transportation
network) to the user benefits under the build alternative (which
assumes the proposed New Starts project is constructed).
[29] FTA officials acknowledged that some benefits may not be counted
under the fixed land use assumptions; however, the magnitude of these
benefits is unknown.
[30] Intermodal Surface Transportation Efficiency Act of 1991 (ISTEA),
Pub. L. No. 102-240, Section 3010, 105 Stat. 2093, Dec. 18, 1991;
Transportation Equity Act for the 21st Century (TEA-21), Pub. L. No.
105-178, Section 3009, 112 Stat. 352, June 9, 1998; and SAFETEA-LU,
Pub. L. No. 109-59, Section 3011, 119 Stat. 1573, Aug. 10, 2005.
[31] TRB, Special Report 288, Metropolitan Travel Forecasting: Current
Practice and Future Direction (2007).
[32] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-172].
[33] Although FHWA has declined to be involved in the request for
proposals, FHWA has worked with FTA to improve the state of the
practice in travel demand modeling and conducts research to advance the
state of the art. For example, the Federal Aid Highway Program has
contributed funds for travel model improvements in recent years,
including funding a significant portion of the TRB study. FHWA
officials strongly agree that models need to be improved, but they
indicated that their tight research budget prevented them from funding
the request for proposals.
[34] TRANSIMS is a set of travel modeling procedures designed to meet
state DOTs' and MPOs' need for more accurate and more sensitive travel
forecasts for transportation planning and emissions analysis. The
amount specified in SAFETEA-LU for TRANSIMS was $2.625 million per
year, but due, in part, to the obligation limitation of FHWA's research
budget, the actual amount was $1.9 million.
[35] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-917].
[36] 72 Fed. Reg. 30907 (June 4, 2007).
[37] Deloitte, New Starts Program Assessment, February 12, 2007.
[38] For more information on the project ratings in the fiscal year
2009 pipeline, see appendix I.
[39] Mobility improvements include two categories of measures: mobility
improvements for the general population and mobility improvements for
transit dependents.
[40] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-172].
[41] Pub. L. No. 110-244, Section 201(d), June 6, 2008.
[42] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-917].
[43] Betterments are generally defined as project elements that are not
essential to the operation of the project but may nevertheless enhance
the operation of the transit service. Examples of such improvements
include additional station entrances to subway stations, substantial
improvements to a station's design beyond the design standards used for
other stations in the system, and changes in the vertical or horizontal
alignment of the project.
[44] Pub. L. No. 105-178, Section 3010, 112 Stat. 357, June 9, 1998.
[45] See GAO, Freight Transportation: Strategies Needed to Address
Planning and Financing Limitations, GAO-04-165 (Washington, D.C.: Dec.
19, 2003) and GAO, Surface Transportation: Many Factors Affect
Investment Decisions, GAO-04-744 (Washington, D.C.: June 30, 2004).
[46] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-744].
[47] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-744].
[48] The participants in GAO's 2007 forum on transforming
transportation policy also maintained that the most important goal of
transportation policy should be to enhance mobility. Further, they
noted that economic development was less important as a goal of federal
transportation policy. See GAO, Highlights of a Forum: Transforming
Transportation Policy for the 21st Century, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-1210SP] (Washington, D.C.:
Sept. 19, 2007).
[49] GAO, Grant Programs: Design Features Shape Flexibility,
Accountability, and Performance Information, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GGD-98-137] (Washington, D.C.: June
22, 1998).
[50] GAO, Surface Transportation: Restructured Federal Approach Needed
for More Focused, Performance-Based, and Sustainable Programs,
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-400] (Washington,
D.C.: Mar. 6, 2008).
[51] In June 2007, FTA replaced the previous three-tiered overall
project rating scale of high, medium, and low with a five-tiered rating
scale of high, medium-high, medium, medium-low, or low as directed by
the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A
Legacy for Users (SAFETEA-LU). Projects requesting less than $25
million in New Starts funding were not evaluated and rated during the
fiscal year 2009 cycle; however, these projects will be evaluated and
rated as "Small Starts" in future cycles, as noted in Section 5309(e)
of SAFETEA-LU.
[52] FTA originally recommended $85 million for final design
activities, but subsequently learned that additional funding was
required for an existing FFGA (Los Angeles Metro Gold Line Eastside
project). As a result, additional funding was allocated to this
project, and less funding was set aside for the final design activities
category.
[53] This proposal is similar to FTA's previous set aside of funding
for other New Starts projects. As in past years, projects that qualify
for this funding must meet the following criteria: (1) received a
medium or higher rating; (2) received a medium or higher cost-
effectiveness rating; and (3) would advance to final design before the
end of the fiscal year.
[54] Due to the 2 percent budget rescission in the fiscal year 2008
Consolidated Appropriations Act (P.L. 110-161), the following four
existing Small Starts projects were not fully funded in fiscal year
2008 as anticipated: Pioneer Parkway EmX Bus Rapid Transit (BRT),
Pacific Highway South BRT, Troost Corridor BRT, and Metro Rapid Bus
System Gap Closure. Consequently, FTA proposed these projects for small
amounts of funding in fiscal year 2009 to complete the agency's
commitment to these projects.
[55] At present, FTA is still working with Portland to develop new
forecasts for its streetcar project because the project did not receive
a medium cost-effectiveness rating. If the Streetcar Loop cannot
achieve a sufficient cost-effectiveness rating by summer 2008, then FTA
will recommend to Congress the reallocation of the project's fiscal
year 2009 Small Starts proposed funding to other emerging Small Starts
projects that demonstrate both the readiness and merit necessary to
meet the administration's goal of funding cost-effective Small Starts
projects.
[56] These comments were accessed through [hyperlink,
http://www.regulations.gov/fdmspublic/component/main], docket number
FTA-2006-25737, accessed November 13, 2007.
[57] See GAO, Highway and Transit Investments: Options for Improving
Information on Projects' Benefits and Costs and Increasing
Accountability for Results, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-05-172] (Washington, D.C.: Jan. 24, 2005) and GAO,
Highlights of a Forum: Transforming Transportation Policy for the 21st
Century, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-1210SP]
(Washington, D.C.: Sept. 19, 2007).
[58] Consumer surplus is a measure of the benefit consumers derive from
using a particular good. It is calculated by taking the difference
between the price consumers are willing to pay and the actual price.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
