Public Transportation
Opportunities Exist to Improve the Communication and Transparency of Changes Made to the New Starts Program Gao ID: GAO-05-674 June 28, 2005The Transportation Equity Act for the 21st Century (TEA-21) and subsequent legislation authorized about $13.5 billion in guaranteed funding for the Federal Transit Administration's (FTA) New Starts program, which is used to select fixed guideway transit projects, such as rail and trolley projects, and to award full funding grant agreements (FFGA). GAO assessed the New Starts process for the fiscal year 2006 cycle. GAO identified (1) the number of projects that were evaluated, rated, and proposed for new FFGAs and the proposed funding commitments in the administration's budget request; (2) changes FTA has made to the New Starts application, evaluation, rating, and oversight processes since the fiscal year 2001 evaluation cycle and how these changes have been communicated to project sponsors; and (3) how FTA developed the measures used to evaluate and rate projects from the criteria outlined in TEA-21 and how those measures are used in the rating process.
For the fiscal year 2006 evaluation cycle, FTA evaluated and rated 27 projects and identified 4 projects that were expected to be ready for new FFGAs before the end of fiscal year 2006 and an additional 6 projects that may be eligible for other funding outside of FFGAs. The administration's fiscal year 2006 budget proposal requests $1.5 billion for the New Starts program, a request similar to that of the past 2 years. FTA has made 16 changes to the New Starts application, evaluation, rating, and oversight processes since the fiscal year 2001 cycle that were primarily intended to make the process more rigorous and systematic. Seven of the 16 changes underwent rulemaking, including providing formal notice to the transit industry and soliciting comment, while 9 changes did not. FTA officials said that these nine changes are consistent with the existing regulations governing the New Starts process or relate to the project development oversight process rather than the evaluation and rating process and, therefore, in their opinion, do not need to undergo formal rulemaking. By not consistently soliciting public opinion, however, FTA is missing an opportunity to obtain stakeholder buy-in, increase the transparency of the New Starts process, and lessen potential difficulties project sponsors face in implementing the changes. Many of the measures FTA uses to evaluate and rate New Starts projects have evolved over time, with industry input, through formal rulemaking and informal efforts, such as workshops and reports. Although both TEA-21 and FTA's New Starts program regulations emphasize the importance of using a multiple-measure approach for evaluating projects, FTA assigns weight to all three financial criteria but only two of the five project justification criteria in developing a project's rating. FTA officials said that they do not use the other three project justification criteria--which are specified in TEA-21--because the measures fail to distinguish among projects. Project sponsors we interviewed offered suggestions for improving all of the project justification measures, and FTA has efforts underway to improve some of the measures.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-05-674, Public Transportation: Opportunities Exist to Improve the Communication and Transparency of Changes Made to the New Starts Program
This is the accessible text file for GAO report number GAO-05-674
entitled 'Public Transportation: Opportunities Exist to Improve the
Communication and Transparency of Changes Made to the New Starts
Program' which was released on June 28, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
June 2005:
Public Transportation:
Opportunities Exist to Improve the Communication and Transparency of
Changes Made to the New Starts Program:
GAO-05-674:
GAO Highlights:
Highlights of GAO-05-674, a report to congressional committees:
Why GAO Did This Study:
The Transportation Equity Act for the 21st Century (TEA-21) and
subsequent legislation authorized about $13.5 billion in guaranteed
funding for the Federal Transit Administration‘s (FTA) New Starts
program, which is used to select fixed guideway transit projects, such
as rail and trolley projects, and to award full funding grant
agreements (FFGA). GAO assessed the New Starts process for the fiscal
year 2006 cycle. GAO identified (1) the number of projects that were
evaluated, rated, and proposed for new FFGAs and the proposed funding
commitments in the administration‘s budget request; (2) changes FTA has
made to the New Starts application, evaluation, rating, and oversight
processes since the fiscal year 2001 evaluation cycle and how these
changes have been communicated to project sponsors; and (3) how FTA
developed the measures used to evaluate and rate projects from the
criteria outlined in TEA-21 and how those measures are used in the
rating process.
What GAO Found:
For the fiscal year 2006 evaluation cycle, FTA evaluated and rated 27
projects and identified 4 projects that were expected to be ready for
new FFGAs before the end of fiscal year 2006 and an additional 6
projects that may be eligible for other funding outside of FFGAs. The
administration‘s fiscal year 2006 budget proposal requests $1.5 billion
for the New Starts program, a request similar to that of the past 2
years.
FTA has made 16 changes to the New Starts application, evaluation,
rating, and oversight processes since the fiscal year 2001 cycle that
were primarily intended to make the process more rigorous and
systematic. Seven of the 16 changes underwent rulemaking, including
providing formal notice to the transit industry and soliciting comment,
while 9 changes did not. FTA officials said that these nine changes are
consistent with the existing regulations governing the New Starts
process or relate to the project development oversight process rather
than the evaluation and rating process and, therefore, in their
opinion, do not need to undergo formal rulemaking. By not consistently
soliciting public opinion, however, FTA is missing an opportunity to
obtain stakeholder buy-in, increase the transparency of the New Starts
process, and lessen potential difficulties project sponsors face in
implementing the changes.
Many of the measures FTA uses to evaluate and rate New Starts projects
have evolved over time, with industry input, through formal rulemaking
and informal efforts, such as workshops and reports. Although both TEA-
21 and FTA‘s New Starts program regulations emphasize the importance of
using a multiple-measure approach for evaluating projects, FTA assigns
weight to all three financial criteria but only two of the five project
justification criteria in developing a project‘s rating. FTA officials
said that they do not use the other three project justification
criteria”which are specified in TEA-21”because the measures fail to
distinguish among projects. Project sponsors we interviewed offered
suggestions for improving all of the project justification measures,
and FTA has efforts underway to improve some of the measures.
Example of a New Starts Project with an Existing FFGA- Bay Area Rapid
Transit Extension to San Francisco International Airport:
[See PDF for image]
[End of figure]
What GAO Recommends:
This report makes a number of recommendations intended to ensure that
the New Starts regulations reflect FTA‘s current evaluation and rating
process and ensure that FTA‘s New Starts evaluation process and
policies are objective, transparent, and follow the spirit of federal
statutes and regulations. FTA officials agreed with the findings and
recommendations in this report.
www.gao.gov/cgi-bin/getrpt?GAO-05-674.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Katherine Siggerud at
(202) 512-2834 or siggerudk@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
FTA Identified Four New Projects for FFGAs and Requested $1.5 Billion
for Fiscal Year 2006:
FTA Has Implemented a Number of Changes to the New Starts Program, Some
Without Project Sponsors' Input:
New Starts Measures Have Evolved Over Time, but Not All Measures Count
Toward a Project's Rating:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Scope and Methodology:
Appendix II: Projects with Existing FFGAs and Projects in Preliminary
Engineering and Final Design in the Fiscal Year 2006 Cycle:
Appendix III: Status of Previous GAO Recommendations for Improving the
New Starts Evaluation Process:
Appendix IV: Key New Starts Provisions Contained in House and Senate
Reauthorization Bills:
Appendix V: FTA's Project Justification Measures for Evaluating and
Rating New Starts Projects:
Appendix VI: GAO Contacts and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: FTA's Criteria for Assigning Overall Project Ratings:
Table 2: Changes to the New Starts Application, Evaluation, Rating, and
Project Development Oversight Processes since the Fiscal Year 2001
Evaluation Cycle:
Table 3: Strengths, Weaknesses, and Other Concerns about New Starts
Measures, as Identified by Project Sponsors:
Table 4: Suggestions for Improving New Starts Measures, as Identified
by Project Sponsors:
Table 5: Projects Contacted for Our Review:
Figures:
Figure 1: New Starts Planning and Development Process:
Figure 2: New Starts Project Evaluation Criteria:
Figure 3: Distribution of New Starts Projects in Preliminary
Engineering and Final Design for Fiscal Year 2000 to 2006 Evaluation
Cycles:
Figure 4: Total New Starts Funding for Fiscal Year 2006 Equals $1.5
Billion:
Figure 5: Criteria and Measures for Evaluating Projects, as Outlined in
TEA-21 and FTA's New Starts Program Regulations:
Figure 6: Weights Used to Determine Project Justification and Financial
Summary Ratings for New Starts Projects:
Abbreviations:
APTA: American Public Transportation Association:
BRT: bus rapid transit:
FFGA: full funding grant agreement:
FTA: Federal Transit Administration:
ISTEA: Intermodal Surface Transportation Efficiency Act of 1991:
LRT: light rail transit:
MOS: minimum operable segment:
TEA-21: Transportation Equity Act for the 21ST Century:
TSUB: Transportation System User Benefits:
Letter June 28, 2005:
The Honorable Richard C. Shelby:
Chairman:
The Honorable Paul S. Sarbanes:
Ranking Minority Member:
Committee on Banking, Housing, and Urban Affairs:
United States Senate:
The Honorable Don Young:
Chairman:
The Honorable James L. Oberstar:
Ranking Democratic Member:
Committee on Transportation and Infrastructure:
House of Representatives:
Much of the federal government's share of new capital investment in
mass transportation since the early 1970s has come through the Federal
Transit Administration's (FTA) New Starts program, which awards full
funding grant agreements (FFGA) for fixed guideway rail, bus rapid
transit, and ferry projects across the country.[Footnote 1] An FFGA
establishes the terms and conditions for federal participation in a
project, including the maximum amount of federal funds available for
the project, which by statute cannot exceed 80 percent of its net cost.
Since fiscal year 1998, the New Starts program has provided state and
local agencies with more than $8.8 billion[Footnote 2] to help design
and construct transit projects throughout the country.
Under the Transportation Equity Act for the 21ST Century (TEA-
21)[Footnote 3] and subsequent amendments and other legislation,
Congress authorized approximately $13.5 billion in New Starts
commitment authority through 2005.[Footnote 4] Even though the level of
program funding is higher than it has ever been, demand for these funds
has also been extremely high. For that reason, TEA-21 directed FTA to
prioritize projects for funding by evaluating, rating, and recommending
potential projects on the basis of specific financial and project
justification criteria--including mobility improvements, cost-
effectiveness, environmental benefits, and operating efficiencies. In
applying these criteria, TEA-21 directed FTA to consider a number of
additional factors, including land use and congestion relief.
Furthermore, TEA-21 required FTA to issue regulations for the
evaluation and rating process, which FTA did in December 2000.
TEA-21 also requires us to report each year on FTA's processes and
procedures for evaluating, rating, and recommending New Starts projects
for federal funding and on the implementation of these processes and
procedures.[Footnote 5] This report discusses (1) the number of
projects that were evaluated, rated, and proposed for new FFGAs for the
fiscal year 2006 evaluation cycle[Footnote 6] and the proposed funding
commitments for New Starts in the administration's fiscal year 2006
budget request; (2) changes that FTA has made to the New Starts
application, evaluation, rating, and oversight processes since TEA-21
and how those changes have been communicated to project sponsors; and
(3) how FTA developed the measures it uses to evaluate and rate
projects from the criteria identified in TEA-21 and how the agency uses
these measures in the evaluation process. To address these objectives,
we reviewed the administration's fiscal year 2006 budget request; the
legislation and regulations governing the New Starts process; and FTA's
annual New Starts reports, reporting instructions, and other program
guidance and documentation. We also interviewed FTA officials and
representatives from the American Public Transportation Association,
the New Starts Working Group,[Footnote 7] and the projects that were
rated in the fiscal year 2006 evaluation cycle.[Footnote 8] In
addition, we attended FTA's meetings with project sponsors--the New
Starts Roundtables--in April and May 2005. We conducted our work from
November 2004 through May 2005 in accordance with generally accepted
government auditing standards. (See app. I for more information about
our scope and methodology.)
Results in Brief:
For the fiscal year 2006 evaluation cycle, FTA evaluated and rated 27
projects and identified 4 that were expected to be ready for new FFGAs
before the end of fiscal year 2006. In addition, FTA identified 6 other
projects as potentially being eligible for funding outside of FFGAs.
The administration is requesting a total of $1.5 billion for the New
Starts program (an amount similar to that requested for the last 2
years). This amount includes $634.6 million for projects with existing
FFGAs and $590 million for projects proposed for new FFGAs. The total
number of projects evaluated and rated has declined slightly every year
since TEA-21, from 42 projects in the fiscal year 2000 evaluation cycle
to 27 projects in the most recent cycle, while the number of
recommended and highly recommended projects has varied from year to
year.
FTA has implemented 16 changes to the New Starts application,
evaluation, rating, and project development oversight processes since
the fiscal year 2001 evaluation cycle, using a variety of communication
methods, but has not consistently sought industry input before
implementing the changes. By not soliciting public opinion, FTA is
missing an opportunity to obtain stakeholder buy-in, increase the
transparency of the New Starts process, and lessen potential
difficulties project sponsors face in implementing the changes.
Although the impetus for each change varied, FTA officials stated that,
in general, all of the changes were intended to make the process more
rigorous, systematic, and transparent. For example, the requirement for
project sponsors to develop a plan for evaluating the impacts of the
project and the accuracy of ridership projections will identify lessons
learned and hold transit agencies accountable for results. Some project
sponsors told us a few of the changes have helped to improve the
program, while others expressed a variety of concerns about the effect
of some changes. For example, seven project sponsors stated that the
"make the case" document helped them focus on the key benefits of the
projects and three said it helped them "sell" the project to local
decisionmakers or the public. Ten project sponsors noted, however, that
FTA did not provide clear guidance on how to develop the "make the
case" document and eight said that they had to produce multiple
iterations of the document. FTA communicates information about changes
to project sponsors through a variety of different methods, including
regulations, reporting instructions, and the agency's Web site. Project
sponsors we interviewed had varying opinions on the effectiveness of
these communication methods but overall found that the Web site was
ineffective because it was difficult to navigate and information was
not consolidated in one location. Seven of the changes that FTA has
made to the New Starts process since the fiscal year 2001 evaluation
cycle underwent rulemaking, including providing formal notice to the
transit industry and soliciting public comment, while nine changes did
not. The Freedom of Information Act requires federal agencies to
publish in the Federal Register notice of changes to programs, and the
Administrative Procedure Act sets out the rulemaking process required
to make changes to agencies' rules and procedures. FTA officials said
that the changes they have made are consistent with the existing
regulations governing the New Starts process and that some of the
changes relate to the project development oversight process rather than
the evaluation and rating process; therefore, in their opinion, the
regulations do not need to be amended. Of the nine changes that did not
undergo rulemaking, six changes--including the administration's new
cost-effectiveness funding recommendation practice that would generally
target funding recommendations to projects that achieve at least a
medium cost-effectiveness rating--were made without FTA providing any
avenues for public review and comment prior to their implementation.
When formal rulemaking is not necessary, there are less formal options
available for soliciting public comment. For example, in March 2005,
FTA solicited public comment on three recent technical changes to the
rating process.
Many of the measures that FTA uses to evaluate projects predate TEA-21
and have evolved over time through an iterative process--involving FTA,
industry, and Congress--through rulemaking, outreach sessions, and
reports. The measures reflect congressional direction to evaluate
projects against a variety of criteria, including mobility
improvements, environmental benefits, operating efficiencies, and cost-
effectiveness, and to consider other issues, such as land use, in the
evaluation of projects. FTA's regulations governing the New Starts
program likewise emphasize a multiple-measure approach. Using a series
of these measures, FTA evaluates projects against statutorily
identified criteria, including three local financial commitment
criteria and five project justification criteria. Each local financial
commitment criterion is counted toward a project's overall rating.
However, only two of the project justification criteria typically count
toward a project's overall rating, despite statements in the New Starts
program regulations that all of the criteria will be used.
Specifically, FTA currently assigns a weight of 50 percent each to the
cost-effectiveness and land use criteria; the other three project
justification criteria are not assigned weights. FTA sought industry
input on whether there should be a weighting system in the last
rulemaking process; however, according to FTA officials, the agency did
not receive input that was useful to inform its policy. Consequently,
the regulations are silent on the weights that should be assigned to
each criterion and do not prohibit FTA from treating various criteria
as more important than others. FTA officials said that the measures for
the other three project justification criteria--mobility improvements,
environmental benefits, and operating efficiencies--do not meaningfully
distinguish among projects, and aspects of the mobility improvements
criterion are already captured in the measure for cost- effectiveness.
Although FTA has made improvements to the measures for land use and
cost-effectiveness, it has not yet been able to identify measures for
the other three project justification criteria that make meaningful
distinctions among projects for the purpose of rating and funding
decisions. FTA officials told us that they will initiate a formal
rulemaking process after the program is reauthorized, at which point
all New Starts policies and procedures will be reevaluated.
This report makes recommendations to the Secretary of Transportation to
ensure that FTA's New Starts regulations reflect its weighting policy,
to improve the measures used to evaluate projects so that all criteria
named in statute can be used to develop a project's rating, to publish
future changes to the New Starts program in the Federal Register, and
to solicit industry comment on changes, through rulemaking or some
other process, as appropriate. To ensure that transit agencies have
clear information on the New Starts program, we are also recommending
that FTA consolidate guidance, reporting instructions, and other New
Starts program information in one location on its Web site. The
Department of Transportation, including FTA, reviewed a draft of this
report. FTA officials generally agreed with the report's findings,
conclusions, and recommendations.
Background:
TEA-21 authorized a total of $36 billion in "guaranteed" funding for a
variety of transit programs, including financial assistance to states
and localities to develop, operate, and maintain transit
systems.[Footnote 9] Under one of these programs, New Starts, FTA
identifies and selects fixed-guideway transit projects for funding--
including heavy, light, and commuter rail; ferry; and certain bus
projects (such as bus rapid transit). FTA generally funds New Starts
projects through FFGAs, which establish the terms and conditions for
federal participation in a New Starts project and also define a
project's scope, including the length of the system and the number of
stations; its schedule, including the date when the system is expected
to open for service; and its cost.
To obtain an FFGA, a project must progress through a local or regional
review of alternatives and meet a number of federal requirements,
including providing information for the New Starts evaluation and
rating process (see fig. 1). As required by TEA-21, New Starts projects
must emerge from a regional, multimodal transportation planning
process. The first two phases of the New Starts process--systems
planning and alternatives analysis--address this requirement. The
systems planning phase identifies the transportation needs of a region,
while the alternatives analysis phase provides information on the
benefits, costs, and impacts of different corridor-level options, such
as rail lines or bus routes. The alternatives analysis phase results in
the selection of a locally preferred alternative--which is intended to
be the New Starts project that FTA evaluates for funding, as required
by statute. After a locally preferred alternative is selected, project
sponsors submit a request to FTA for entry into the preliminary
engineering phase.[Footnote 10] Following completion of preliminary
engineering and federal environmental requirements, the project may be
approved by FTA to advance into final design,[Footnote 11] after which
the project may be approved by FTA for an FFGA and proceed to
construction, as provided for in statute. FTA oversees grantee
management of projects from the preliminary engineering phase through
construction and evaluates the projects for advancement into each phase
of the process, as well as annually for the New Starts report to
Congress. We have recognized the New Starts program as a good model
that the federal government could use for approving other
transportation projects.
Figure 1: New Starts Planning and Development Process:
[See PDF for image]
[End of figure]
To help inform administration and congressional decisions about which
projects should receive federal funds, FTA assigns ratings based on a
variety of financial and project justification criteria, as defined by
its program regulations, and then assigns an overall rating. These
criteria are identified in TEA-21 and reflect a broad range of benefits
and effects of the proposed project, such as cost-effectiveness, as
well as the ability of the project sponsor to fund the project and
finance the continued operation of its transit system (see fig. 2).
Projects are rated at several points during the New Starts process--as
part of the evaluation for entry into preliminary engineering and final
design, and yearly for inclusion in the New Starts annual report.
Figure 2: New Starts Project Evaluation Criteria:
[See PDF for image]
[A] Other factors can include environmental justice and equity issues,
economic development initiatives, innovative financing, etc.
[End of figure]
FTA assigns the proposed project a rating of high, medium-high, medium,
low-medium, or low for each criterion, then assigns a summary rating
for local financial commitment and project justification. Finally, FTA
develops an overall project rating of "highly recommended,"
"recommended," "not recommended," or "not rated." (See table 1 for the
criteria FTA uses to evaluate projects.) The exceptions to this process
are statutorily "exempt" projects, which are those that request less
than $25 million in New Starts funding. These projects are not required
to submit project justification information--although FTA encourages
them to do so--and do not receive ratings from FTA; thus, the number of
projects in preliminary engineering or final design may be greater than
the number of projects evaluated and rated by FTA.
Table 1: FTA's Criteria for Assigning Overall Project Ratings:
Overall rating category: Highly recommended;
Criteria: Requires at least a medium-high for both the financial and
project justification summary ratings.
Overall rating category: Recommended;
Criteria: Requires at least a medium for both the financial and project
justification summary ratings.
Overall rating category: Not recommended;
Criteria: Assigned to projects not rated at least medium for both the
financial and project justification summary ratings.
Overall rating category: Not rated;
Criteria: Indicates that insufficient information was submitted or that
FTA has serious concerns about the information submitted because the
underlying travel forecasting assumptions used by the project sponsor
may have inaccurately represented the benefits of the project.
Source: FTA.
[End of table]
As required by statute, the administration uses the FTA evaluation and
rating process, along with the stage of development of New Starts
projects, to decide which projects to recommend to Congress for
funding.[Footnote 12] Although many projects receive an overall rating
of "recommended" or "highly recommended," only a few are proposed for
FFGAs in a given fiscal year. FTA proposes "recommended" or "highly
recommended" projects for FFGAs when it believes that the projects will
be able to meet certain conditions during the fiscal year that the
proposals are made. These conditions include the following:
* All non-New Starts funding must be committed and available for the
project.
* The project must be in the final design phase and have progressed to
the point where uncertainties about costs, benefits, and impacts (e.g.,
environmental or financial) are minimized.
* The project must meet FTA's tests for readiness and technical
capacity, which confirm there are no cost, project scope, or local
financial commitment issues remaining.
FTA Identified Four New Projects for FFGAs and Requested $1.5 Billion
for Fiscal Year 2006:
Of the 34 projects in preliminary engineering or final design for the
fiscal year 2006 cycle, 27 were evaluated and rated and 7 were
statutorily exempt from the rating process. Four projects were
recommended for funding with the expectation that they would be ready
for new FFGAs before the end of fiscal year 2006, and an additional 6
projects were identified as potentially receiving a recommendation for
New Starts funding outside of FFGAs. The administration's fiscal year
2006 budget proposal requests a total of $1.5 billion for the New
Starts program, an amount similar to requests for the past 2 fiscal
years. (See app. II for the administration's 2006 budget proposal and
FTA's project ratings.)
FTA Evaluated and Rated 27 Projects and Proposed 4 for FFGAs in Fiscal
Year 2006:
FTA's Annual Report on New Starts: Proposed Allocations of Funds for
Fiscal Year 2006 ("annual report") listed a total of 34 projects in
preliminary engineering and final design, and FTA evaluated and rated
27 of them.[Footnote 13] Seven were statutorily exempt from being rated
because they requested less than $25 million in New Starts funding. Of
the 27 projects that were rated, 2 were highly recommended, 12 were
recommended, 8 were not recommended, and 5 were designated "not rated."
In its annual report, FTA said that projects were designated as "not
rated" because they did not submit the required information or because
of FTA's continuing concerns about the reliability of the
transportation benefits forecast for these projects. According to FTA,
a principal source of these concerns was inconsistent assumptions used
in defining the baseline alternative and the proposed New Starts
project, making it difficult to isolate the impacts of the proposed
project. In some cases, the local travel demand models were
inconsistent with FTA guidance and good planning practice. FTA is
currently working with the sponsors of these projects to improve the
forecasts.
As shown in figure 3, the combined number of recommended and highly
recommended projects declined sharply from 27 in the fiscal year 2003
evaluation cycle to 14 in the fiscal year 2004 evaluation cycle, while
the combined number of not recommended and not rated projects rose from
6 to 18. As we previously reported, this was primarily due to
difficulties project sponsors encountered when implementing the new
cost-effectiveness measure--the Transportation System User Benefits
(TSUB) measure[Footnote 14]--and adjusting to FTA's preference policy
that favors projects that seek a federal New Starts share of no more
than 60 percent of the total project costs.[Footnote 15] According to
FTA, the information that was provided by project sponsors in their
reports on TSUB highlighted previously unknown problems with many local
models that forecast travel demand. FTA first incorporated both of
these changes beginning with the fiscal year 2004 evaluation cycle.
Project ratings generally improved (i.e., there were more projects with
recommended ratings and fewer with not recommended ratings) in the
fiscal year 2005 evaluation cycle as project sponsors improved their
financial plans, grew more comfortable with the new cost-effectiveness
measure, and made corrections and improvements to their models that
forecast travel demand, according to FTA.[Footnote 16] While the
combined number of not rated and not recommended projects was
approximately the same in the fiscal year 2005 and 2006 evaluation
cycles, the number of projects receiving ratings of at least
recommended has decreased slightly again. Unlike in previous years,
however, there was no obviously identifiable reason for this change,
except for the fact that some projects moved out of the ratings pool
and into construction, according to FTA.
Figure 3: Distribution of New Starts Projects in Preliminary
Engineering and Final Design for Fiscal Year 2000 to 2006 Evaluation
Cycles:
[See PDF for image]
Note: The pool of projects evaluated and rated each year changes, as
some projects from the previous year receive FFGAs and begin
construction, while new projects enter preliminary engineering and are
evaluated and rated for the first time.
[End of figure]
The fiscal year 2006 rating cycle saw the smallest total number of
projects in preliminary engineering and final design during the TEA-21
period. The number of projects evaluated and rated has decreased
slightly every year, from 42 projects in the fiscal year 2000 rating
cycle to 27 projects in the most recent cycle.[Footnote 17] During this
same time frame, the number of exempt projects grew steadily through
the fiscal year 2004 cycle but has been declining ever since. FTA
officials suggested that these trends could be the result of a
combination of factors. First, many of the projects that TEA-21
authorized have worked their way through the New Starts process and
obtained FFGAs, and thus are not subject to the annual evaluation and
rating process. At the same time, new projects entered preliminary
engineering each year and were rated for the first time. FTA officials
speculated that additional projects would be included in
reauthorization legislation and would enter the New Starts pipeline
over the course of the reauthorization period. Second, FTA has
increased the level of scrutiny it applies to projects attempting to
advance from alternatives analysis to preliminary engineering to help
ensure that only the strongest projects enter the New Starts pipeline.
Third, FTA has worked with project sponsors to reduce the number of
inactive projects in the New Starts pipeline. Fourth, since the time
TEA-21 was enacted, some state and local transportation agencies have
been feeling the impact of local budget constraints, making it more
difficult to secure local funding for proposed New Starts projects.
This could be exacerbated by FTA's policy of favoring projects that
seek a federal New Starts share of no more than 60 percent of the total
project costs. As we reported in 2004, several project sponsors told us
that FTA's push for a lower federal New Starts share would likely
affect their decision to advance future transit projects.[Footnote 18]
Therefore, we recommended that FTA examine the impact of the preference
policy on projects in the evaluation process. FTA is considering how to
conduct such an examination.
FTA's evaluation process informed the administration's recommendation
to fund four projects that are expected to be ready for new FFGAs
before the end of fiscal year 2006, including Charlotte, South Corridor
Light Rail Transit; New York, Long Island Rail Road East Side Access;
Phoenix, Central Phoenix/East Valley Light Rail Transit Corridor; and
Pittsburgh, North Shore Light Rail Transit Connector. The total capital
cost of these four projects is estimated to be $9.9 billion, of which
the total federal New Starts share is expected to be $3.6 billion. FTA
executed an FFGA for the Phoenix project in January 2005 and for the
Charlotte project in May 2005. The other two projects are expected to
be ready for FFGAs before the end of fiscal year 2006. According to
FTA, the remaining projects that received overall ratings of
recommended or highly recommended do not yet pass FTA's readiness tests
for FFGAs.
The administration also proposed reserving $158.6 million in New Starts
funding for final design and early construction activities for as many
as six "other projects," including San Diego, Mid-Coast Light Rail
Transit Extension; Denver, West Corridor Light Rail Transit; New York,
Second Avenue Subway; Washington County (Oregon), Wilsonville to
Beaverton Commuter Rail; Dallas, Northwest-Southeast Light Rail; and
Salt Lake City, Weber County to Salt Lake City Commuter Rail. These six
projects were in or nearing final design, received overall highly
recommended or recommended ratings, and had cost-effectiveness ratings
above "low." According to FTA officials, no other projects met these
criteria. The annual report did not specify amounts for particular
projects to ensure that the project is moving forward as anticipated
prior to making specific funding recommendations to Congress, according
to FTA officials, because projects may encounter unexpected financial
or other obstacles that slow their progress. For example, FTA told us
that the sponsor for one of the projects is considering a significant
expansion of the project scope, which would put it back in preliminary
engineering and render it ineligible for the funds FTA proposed
reserving for the "other projects" for fiscal year 2006. Reserving
funds for these projects without specifying a particular amount for any
given project will allow the administration to make "real time" funding
recommendations when Congress is making appropriations decisions. FTA
does not anticipate that all of the six projects will be recommended
for funding in fiscal year 2006.
Administration's Proposed Fiscal Year 2006 Budget Requests Similar
Amount of Funding to Previous Years:
The administration's fiscal year 2006 budget proposal requests that
$1.5 billion be made available for the New Starts program, an amount
similar to that requested in the last two fiscal years. Figure 4
illustrates the specific budget allocations the administration has
proposed for fiscal year 2006, including the following:
* $634.6 million would be allocated among the 16 projects with existing
grant agreements,
* $590 million would be allocated to four projects expected to be
proposed for new FFGAs,
* $158.6 million would be allocated to as many as six "other" projects
to continue project development, and:
* $122.5 million is reserved to be allocated among projects in
preliminary engineering, at the discretion of Congress and as provided
in law.[Footnote 19]
Figure 4: Total New Starts Funding for Fiscal Year 2006 Equals $1.5
Billion:
[See PDF for image]
Note: FTA is authorized to use up to 1 percent of amounts made
available for the New Starts program for project management oversight
activities. TEA-21 requires that specified amounts of New Starts funds
be set aside annually for projects in Alaska and Hawaii, for new fixed
guideway systems and extensions to existing systems that are ferry
boats or ferry terminal facilities or that are approaches to ferry
terminal facilities.
[End of figure]
FTA had approximately $2.2 billion in commitment authority--that is,
the amount of funding authorized for New Starts projects--remaining as
of May 2005. TEA-21 and subsequent amendments and legislation provided
FTA the authority to make about $13.5 billion in funding commitments
for New Starts projects. Surface transportation programs, including the
New Starts program, were scheduled to expire in September 2003, but
have subsequently been extended through June 2005.[Footnote 20]
According to FTA officials, the commitment authority for fiscal year
2006 and beyond will be addressed in the next surface transportation
authorization legislation. However, FTA officials told us that there
will not be sufficient commitment authority remaining to execute all
four proposed FFGAs until additional commitment authority is provided
through congressional authorization.
FTA Has Implemented a Number of Changes to the New Starts Program, Some
Without Project Sponsors' Input:
FTA has made 16 changes to the New Starts application, evaluation,
rating, and project development oversight processes since the fiscal
year 2001 evaluation cycle--the first full evaluation and rating cycle
after the enactment of TEA-21. FTA has used a variety of written and
electronic methods to communicate information about these changes,
although it primarily relies on reporting instructions, roundtables,
and workshops. For nine of these changes, FTA did not publish
information about the change in the Federal Register or institute a
rulemaking process and for six of these nine changes did not provide
any avenues for public review and comment prior to implementing the
changes. FTA has said that all of the changes it has made are
consistent with the evaluation framework outlined in the existing
regulations governing the New Starts process. Some project sponsors we
interviewed thought certain changes helped to improve the process;
however, a considerable number of project sponsors expressed concern
that they did not have an opportunity to comment on many of the changes
before they were implemented and have experienced considerable
challenges while attempting to incorporate some of the changes. For
example, some changes have required project sponsors to devote
additional agency resources, as well as time.
FTA Has Made 16 Changes Intended to Improve the Application,
Evaluation, Rating, and Project Development Oversight Processes since
the Fiscal Year 2001 Evaluation Cycle:
We identified 16 changes that FTA made to the New Starts application,
evaluation, rating, and project development oversight processes since
the fiscal year 2001 evaluation cycle.[Footnote 21] These changes range
from requiring that projects undergo a risk assessment to instituting
new practices for funding recommendations. (See table 2 for complete
list of changes.) For example, FTA made two significant changes to the
evaluation and rating process for the fiscal year 2004 evaluation
cycle. First, FTA implemented the TSUB measure as a variable in the
calculation of cost-effectiveness and mobility improvements.[Footnote
22] The new measure is intended to calculate the change in the amount
of travel time and costs that people incur for taking a trip. This is a
more comprehensive measure than the old "cost per new rider" measure
because it takes into account a broader set of benefits to transit
riders, including new and existing transit riders. Second, in response
to language contained in appropriations committee reports, FTA
instituted a preference policy favoring projects that seek a federal
New Starts share of no more than 60 percent of the total project cost.
As shown in table 2, FTA implemented changes for a variety of reasons,
including simplifying the New Starts process and focusing more on
results and performance. Although the impetus for each change varied,
FTA officials stated that, in general, all the changes were intended to
make the process more rigorous, systematic, and transparent. For
example, the requirement for project sponsors to develop a plan for
evaluating the impacts of the project and the accuracy of travel
forecasts--that is, a Before and After study--will identify lessons
learned and hold transit agencies accountable for results. In our
previous work we have commented on a number of these changes and made
some recommendations for improving the New Starts process. (See app.
III for a list of recommendations from previous reports.)
Table 2: Changes to the New Starts Application, Evaluation, Rating, and
Project Development Oversight Processes since the Fiscal Year 2001
Evaluation Cycle:
Change: Additional subfactors added to the land use criterion;
Brief description of change: Several additional "statutory
considerations" were added to the land use criterion, including the
cost of sprawl, infrastructure cost savings due to compact land use,
and population density and current transit ridership in a corridor;
Method and date introduced: Transportation Equity Act for the 21st
Century (TEA-21)/New Starts regulations, June 1998/Dec. 2000;
Evaluation cycle effective: Fiscal year (FY) 2003;
Reason for change: Required by TEA- 21;
Published in Federal Register? Yes;
Formal opportunity for comment? Yes.
Change: Baseline alternative introduced;
Brief description of change: FTA eliminated the requirement for the
separate no-build and transportation system management alternatives,[A]
and instead requires that the proposed New Start be evaluated against a
single "baseline alternative" (i.e., the best that can be done without
the New Starts investment);
Method and date introduced: New Starts regulations, Dec. 2000;
Evaluation cycle effective: FY 2003;
Reason for change: Simplify New Starts process and address project
sponsor concerns about undue reporting burden;
Published in Federal Register? Yes;
Formal opportunity for comment? Yes.
Change: Pedestrian mobility formally incorporated into land use
criterion;
Brief description of change: Pedestrian mobility had been a component
of FTA's land use evaluation, as described in FTA guidance issued each
year. However, the regulations formally incorporated an element for
pedestrian mobility into the land use criterion;
Method and date introduced: New Starts regulations, Dec. 2000;
Evaluation cycle effective: FY 2003;
Reason for change: Formalize agency practice;
Published in Federal Register? Yes;
Formal opportunity for comment? Yes.
Change: "Employment near stations" measure added to mobility
improvements criterion;
Brief description of change: FTA added a new factor to calculate
destinations for jobs within a half-mile radius of boarding points on
the new system, complementing the existing factor that measures low-
income households within a half-mile radius of boarding points;
Method and date introduced: New Starts regulations, Dec. 2000;
Evaluation cycle effective: FY 2003;
Reason for change: Address industry concerns that FTA was undervaluing
destination trips;
Published in Federal Register? Yes;
Formal opportunity for comment? Yes.
Change: Decision rule for financial summary rating;
Brief description of change: FTA instituted a decision rule that
required projects to receive at least a "medium" rating for both
capital and operating plans to get an overall "recommended" rating;
Method and date introduced: New Starts regulations, Dec. 2000;
Evaluation cycle effective: FY 2003;
Reason for change: Emphasize both capital and operating plans;
Published in Federal Register? Yes;
Formal opportunity for comment? Yes.
Change: Before and After study required[B];
Brief description of change: Project sponsors seeking a full funding
grant agreement (FFGA) for their New Starts project must submit to FTA
a plan for the collection and analysis of information leading to the
identification of the impacts of the project and the accuracy of the
forecasts that were prepared during project planning and development;
Method and date introduced: New Starts regulations, Dec. 2000;
Evaluation cycle effective: FY 2003;
Reason for change: Overall government focus on performance and results;
Published in Federal Register? Yes;
Formal opportunity for comment? Yes.
Change: Transportation System User Benefits (TSUB) measure introduced;
Brief description of change: FTA revised its cost-effectiveness and
mobility improvements criteria by adopting the TSUB measure that
includes benefits for both new and existing transit system riders;
Method and date introduced: New Starts regulations, Dec. 2000;
Evaluation cycle effective: FY 2004;
Reason for change: Address industry's concerns about cost per new rider
index;
Published in Federal Register? Yes;
Formal opportunity for comment? Yes.
Change: Project justification criteria weights changed;
Brief description of change: The cost-effectiveness and land use
criteria were each weighted 50%. In the previous rating cycle, FTA
attempted to use all five project justification criteria in the rating
process;
Method and date introduced: Reporting instructions,[C] June 2003;
Evaluation cycle effective: FY 2004;
Reason for change: Other criteria's measures do not meaningfully
distinguish among projects;
Published in Federal Register? No;
Formal opportunity for comment? No.
Change: 60% preference policy instituted;
Brief description of change: FTA instituted a preference policy in its
ratings process favoring current and future projects that do not
request more than a 60% federal share. To achieve this, FTA changed its
criterion related to capital finance plans to give projects seeking a
federal share greater than 60% a "low" financial rating;
Method and date introduced: Annual report for FY 2004 evaluation cycle;
Evaluation cycle effective: FY 2004;
Reason for change: FY 2004 Appropriations Conference Report;
Published in Federal Register? No;
Formal opportunity for comment? No.
Change: Risk assessments introduced[B];
Brief description of change: The Risk Assessment is an FTA management
and oversight tool intended to identify the issues that could affect
schedule or cost, as well as the probability that a cost estimate will
be met;
Method and date introduced: New Starts and FTA construction
roundtables,[D] April/May 2003;
Evaluation cycle effective: FY 2005;
Reason for change: Implemented for project management oversight;
Published in Federal Register? No;
Formal opportunity for comment? No.
Change: "Make the case" document required[B];
Brief description of change: FTA requires all project sponsors to
submit a three-page narrative that justifies why the New Starts project
is the best possible alternative and why it is needed;
Method and date introduced: Reporting instructions,[C] June 2003;
Evaluation cycle effective: FY 2005;
Reason for change: To have a short document that describes the
individual merits of a project from the community's perspective;
Published in Federal Register? No;
Formal opportunity for comment? No.
Change: Travel forecasts requested in advance of application deadline;
Brief description of change: FTA encouraged (but did not require)
grantees to submit their travel forecasts by June 30, 2004, in advance
of the Aug. 20 deadline for the rest of the information;
Method and date introduced: Reporting instructions,[C] April 2004;
Evaluation cycle effective: FY 2006;
Reason for change: To allow technical assistance to be provided in time
for the project to be rated in the upcoming annual report;
Published in Federal Register? No;
Formal opportunity for comment? No.
Change: Cost-effectiveness practice[B];
Brief description of change: Generally, the administration will target
its funding recommendations to projects that achieve a cost-
effectiveness rating of medium or higher to be recommended for funding.
Previously, the administration would recommend projects for funding
that had a low-medium cost- effectiveness rating, if they met all other
criteria;
Method and date introduced: New Starts Workshop[E] and FTA "Dear
Colleague" letter,[F] March 2005;
Evaluation cycle effective: FY 2007;
Reason for change: Address concerns raised by the DOT Inspector
General, Congress, and Office of Management and Budget;
Published in Federal Register? No;
Formal opportunity for comment? No.
Change: Standard cost categories required;
Brief description of change: FTA updated the useful life assumptions
for various categories of assets and is requiring project sponsors to
report costs in standard categories;
Method and date introduced: New Starts Workshop[E] and FTA "Dear
Colleague" letter,[F] March 2005;
Evaluation cycle effective: FY 2007;
Reason for change: To be consistent with useful life estimates;
Published in Federal Register? No;
Formal opportunity for comment? Yes.
Change: Cost-effectiveness rating breakpoints adjusted for inflation;
Brief description of change: FTA adjusted cost-effectiveness rating
breakpoints for inflation using the Gross Domestic Price Index;
Method and date introduced: New Starts Workshop[E] and FTA "Dear
Colleague" letter,[F] March 2005;
Evaluation cycle effective: FY 2007;
Reason for change: Cost-effectiveness breakpoints were first
established in 2002;
Published in Federal Register? No;
Formal opportunity for comment? Yes.
Change: 2030 planning horizon permitted;
Brief description of change: Project sponsors will be permitted to use
either a 2025 or 2030 planning horizon for estimating the project's
costs and benefits to maintain consistency with the horizon year used
by the local metropolitan planning organization;
Method and date introduced: New Starts Workshop[E] and FTA "Dear
Colleague" letter,[F] March 2005;
Evaluation cycle effective: FY 2007;
Reason for change: To maintain consistency with what local metropolitan
planning organizations are using;
Published in Federal Register? No;
Formal opportunity for comment? Yes.
Sources: GAO and FTA.
[A] The transportation system management alternative is equivalent to
the New Starts baseline alternative for most New Starts projects.
[B] FTA officials identified these changes as involving project
development oversight that do not affect project ratings.
[C] FTA uses reporting instructions to provide project sponsors with
technical and procedural assistance on the application and reporting of
New Starts criteria.
[D] FTA uses New Starts roundtables to share information and
experiences with project sponsors, identify and discuss common issues,
and generate suggestions for improving the planning and development
process.
[E] The American Public Transportation Association typically hosts an
annual legislative conference at which FTA conducts an information
session on the New Starts program.
[F] The Dear Colleague letter is periodically sent from the FTA
Administrator to project sponsors regarding issues associated with the
New Starts program.
[End of table]
Some project sponsors we interviewed thought that certain changes
helped to improve the process. For instance, seven project sponsors
stated that the "make the case" document helped them focus on the key
benefits of the projects and three said it helped them "sell" the
project to local decisionmakers or the public. In addition, many
project sponsors acknowledged that the TSUB measure is more
comprehensive than the old "cost per new rider" measure. However,
project sponsors expressed a variety of concerns about the effects the
changes had on the steps required to complete the application process
and about FTA's implementation of the changes. For instance, for the
"make the case" document, 15 project sponsors stated that FTA did not
create clear expectations, 10 noted that FTA prepared no written
guidance, and 5 indicated that FTA did not provide specific examples or
templates.[Footnote 23] A few of these project sponsors also stated
that they did not feel like they understood what FTA wanted for the
"make the case" document, and eight said that they had to produce
multiple iterations of the document. Twelve project sponsors also
mentioned they were not clear about how FTA was using the document. In
addition, when asked what effect the implementation of the TSUB measure
had on their project, 13 of 26 project sponsors said it made the
application process more expensive, and 20 of 26 said that this measure
required them to spend significantly more time to complete the
application. Similar comments were made by project sponsors about other
changes, including the risk assessment and Before and After study
requirements.[Footnote 24] For example, when asked what effect the risk
assessment requirement has had on their project, 4 of the 26 project
sponsors said it resulted in a more rigorous or systematic evaluation
and rating process, 7 said it made the process more expensive, and 5
said it delayed their project. Three project sponsors noted that each
of the individual requirements add to the overall workload and
eventually result in the application process becoming a full-time
project. However, some project sponsors observed that as they become
more familiar with each change and as FTA issues more guidance, such as
for using FTA's software to calculate the TSUB value, the change has
become less burdensome.
In Spring 2005 FTA Introduced New Cost-Effectiveness Practice and
Technical Changes to the Cost-Effectiveness Calculation:
In March 2005, FTA announced a new practice for the New Starts program
whereby the administration will generally target funding
recommendations to projects able to achieve at least a medium or higher
cost-effectiveness rating. FTA announced this change through a "Dear
Colleague" letter sent to FTA grantees, including current New Starts
project sponsors. The letter was also posted on the home page of FTA's
Web site. The administration's previous policy had been to recommend
projects for funding that received at least a medium-low cost-
effectiveness rating, provided that they met all other criteria and
project readiness requirements. In the letter to project sponsors, FTA
explained that the impetus for change was the concerns that GAO, the
Office of Management and Budget, the Department of Transportation's
Inspector General, and Congress raised about recommending projects for
funding that had medium-low cost-effectiveness ratings.[Footnote 25] In
the letter, and in subsequent communication to project sponsors, FTA
also stated that the practice applies to a recommendation for funding
and not to the way the project's overall rating is determined. For
example, a project can still receive an overall recommended project
rating with a medium-low cost-effectiveness rating and can advance from
preliminary engineering to final design with this rating; however, as a
general rule, the project would not be recommended for funding by the
administration. FTA has said that this new practice will help it to
further prioritize and distinguish among projects for federal funding,
which is important given current fiscal challenges and the resulting
need to maximize the benefit of every federal dollar invested in
transportation.
According to FTA officials, the four New Starts projects with
anticipated FFGAs for the fiscal year 2006 cycle (see app. II) will not
be affected by the new practice. However, FTA has said that the six New
Starts projects that are categorized as "other projects" in the fiscal
year 2006 annual report--and therefore eligible for a portion of the
$158.6 million in New Starts funds FTA reserved--will be subject to the
new cost-effectiveness funding recommendation practice and will
continue to be subject to this practice when they apply for an FFGA.
According to FTA, two of the six "other projects" currently do not meet
this standard, and FTA is working with them to improve their cost-
effectiveness rating. Six project sponsors we interviewed expressed
concern as to whether they would be able to make the necessary
modifications to their projects in order to earn a higher cost-
effectiveness rating by the next evaluation cycle--which begins in
August 2005. Also, seven project sponsors commented that FTA puts too
much emphasis on the cost-effectiveness measure, and four indicated
that the new cost-effectiveness practice increases this emphasis. FTA
officials stated that FTA and the administration consider more than
just cost-effectiveness in making funding recommendations, noting that
every project that received a not recommended rating in the last 2
years did so because of its poor financial summary rating.
In addition to the new cost-effectiveness practice in the March "Dear
Colleague" letter, FTA also proposed five technical changes to the way
cost-effectiveness is calculated and asked for industry comment on
these proposed changes by April 1, 2005. All comments were posted on
the Department of Transportation's online docket[Footnote 26] and were
available for public review. The proposed changes included (1)
adjusting the cost-effectiveness rating breakpoints (i.e., low, low-
medium, medium, medium-high, high) for inflation, and possibly applying
a regional index in an effort to address cost differences across the
country; (2) permitting project sponsors to utilize either a 2025 or
2030 planning horizon to be consistent with metropolitan planning
organizations' regional planning processes; (3) permitting standardized
costs and the proposed adjustments to useful life estimates, which
clarify and lengthen these estimates for a number of assets; (4)
permitting modal constants for new guideway modes as a means of
enabling travel models to estimate the effect of improvements to
transit service quality beyond the time and cost measures already
accounted for in the travel models (such as comfort and reliability);
and (5) excluding some soft costs (e.g., administrative expenses) from
the calculation of annualized capital costs for the purpose of
calculating cost-effectiveness. FTA officials told us that they wanted
to obtain industry comment on the proposed changes and to use this
feedback to help decide which proposed changes to adopt. Officials from
FTA also said the deadline for industry comments was driven by the
reporting deadlines for the New Starts annual evaluations and the need
to promptly release reporting instructions for the News Starts program.
On the basis of FTA's review of the proposed changes and project
sponsor responses, FTA announced in an April 29TH "Dear Colleague"
letter its decision to incorporate three of the five proposed changes
for the fiscal year 2007 cycle--adjusting rating breakpoints for
inflation, permitting the 2030 design year forecast, and permitting the
use of the standardized cost categories' useful life assumptions to
calculate annualized capital costs for the purpose of calculating cost-
effectiveness. FTA stated that further research is needed on the other
two proposed changes. FTA incorporated the three technical changes into
its fiscal year 2007 reporting instructions, which were released to
project sponsors on May 3, 2005.
FTA Uses Multiple Methods to Communicate Information about Changes:
FTA communicates information about changes to project sponsors through
a variety of different methods, including annual reporting
instructions, "Dear Colleague" letters to project sponsors,
conversations with project sponsors, roundtables and workshops with FTA
officials and project sponsors, and the FTA Web site. FTA officials
indicated that they most heavily rely on regulations, reporting
instructions, and roundtables and workshops to communicate information
about changes to the New Starts application, evaluation, rating, and
project development oversight processes. In particular, FTA officials
told us that they use the workshops as a two-way communication vehicle,
during which they can explain the New Starts application and evaluation
process to project sponsors. The workshops are usually held two to four
times a year in conjunction with a transit industry conference. For
example, during the American Public Transportation Association (APTA)
legislative conference in early March 2005, FTA officials held a
workshop to discuss the New Starts process as well as the
administration's new cost-effectiveness funding recommendation
practice. The agency also holds two New Starts roundtables each year to
explain the application and evaluation process and allow project
sponsors the opportunity to have an open discussion with FTA officials
as well as share information on best practices. FTA used this year's
New Starts roundtables in New York and San Francisco to, among other
things, respond to questions about the recently introduced changes.
FTA's annual reporting instructions also describe changes to the
application and evaluation process, in addition to providing guidance
to project sponsors on preparing their New Starts submittal. For
instance, in the reporting instructions for the New Starts report for
the fiscal year 2005 evaluation cycle, FTA introduced the change to the
weighting system used to calculate the project justification rating.
Project sponsors told us that they generally learn about changes
through one or more of FTA's communication methods, although they have
varying views on the effectiveness of the different methods. Most
project sponsors we interviewed said that they typically view the
roundtables and workshops and conversations with FTA officials as
generally or very effective methods for learning about changes.
However, some concerns were also raised about these communication
methods. First, three project sponsors stated that they feel compelled
to attend all of FTA's workshops and roundtables in order to keep up
with the changes to the New Starts process because these meetings are
the primary methods FTA uses to introduce changes. Others noted that
they have received inconsistent information about changes depending on
the source. As a result, 18 of 26 project sponsors stated they prefer
to obtain information about the changes directly from FTA officials--
and 11 of these project sponsors stated that they prefer to receive the
information in writing for documentation purposes. Eleven of the 26
project sponsors we interviewed said that FTA's Web site was generally
or very ineffective as a method for communicating information about
changes. Project sponsors cited a variety of challenges in using the
Web site as a source of information about changes to the New Starts
program. For example, some project sponsors stated the Web site was
difficult to navigate and had a poor search engine, while others stated
they had difficulty finding information about the New Starts program
and that in some cases the information they retrieved from the Web site
was not up to date. Several project sponsors indicated that maintaining
a central repository for all information related to the New Starts
program on FTA's Web site would be helpful.
FTA Has Not Consistently Used the Public Notice or Rulemaking Process
to Introduce Changes or Solicit Industry Comment on Proposed Changes:
Of the 16 changes that FTA has made to the New Starts application,
evaluation, rating, and project development oversight processes since
the fiscal year 2001 evaluation cycle, seven underwent rulemaking,
including providing formal notice to the transit industry and
soliciting public comment, while nine changes did not (see table 2).
The Freedom of Information Act requires federal agencies to publish in
the Federal Register notice of changes to programs and the
Administrative Procedure Act sets out the rulemaking process--which
would include notifying the public and soliciting and considering
comments, among other things--required to make changes to agencies'
rules and procedures. FTA last undertook rulemaking for the New Starts
program in 1999 at the direction of TEA-21 and issued regulations in
December 2000.[Footnote 27] Officials from FTA told us that they have
not amended their regulations to incorporate the nine changes made to
the New Starts process since that time because, in their opinion, the
changes are within the framework of the current regulations. FTA
officials also said that some of the changes--such as the risk
assessment requirement--involve project development oversight and do
not affect the evaluation and rating process; therefore, in their view,
the changes do not need to be included in program regulations. However,
by making changes outside of the regulatory process, FTA has missed an
opportunity to obtain formal public input on the proposed changes,
which would increase the transparency of the agency's decision-making
process and ensure that the views of project sponsors and other
interested parties are considered. There are a range of options
available to FTA for obtaining industry input, from the more formal
rulemaking process to less formal ways of soliciting comment. In those
instances where FTA determines that a formal rulemaking process is
unnecessary, it could provide project sponsors an informal opportunity
to review and comment on any substantive changes proposed for the New
Starts program, as FTA recently did when it solicited public comment on
proposed technical changes to the rating process.
Of the nine changes that were not published in the Federal Register and
did not undergo rulemaking, six were made without providing other
avenues for public review and comment prior to their implementation.
For example, FTA did not seek industry input before implementing the
risk assessment requirement, project justification criteria weights,
and the "make the case" changes to the document. The limited
opportunities to review and comment on proposed changes have resulted
in implementation problems, according to some project sponsors. In
addition, FTA did not provide project sponsors the opportunity to
comment on the administration's recently announced cost-effectiveness
funding recommendation practice. Many project sponsors expressed
concern about the way FTA has developed and implemented changes to the
New Starts process. For example, 14 of the 26 project sponsors with
whom we spoke said that they have generally not been given an
opportunity to offer input into prospective changes before they are
implemented. Many project sponsors noted that the March "Dear
Colleague" letter was a step in the right direction in that it was one
of the first times since the last rulemaking that FTA had sought their
input on proposed changes. However, five project sponsors commented
that FTA did not give them sufficient time to review and comment on the
proposed changes. In addition, FTA did not publish these changes in the
Federal Register and provide at least 30 days' notice, the minimum time
typically required when changes are subject to the Administrative
Procedure Act. Congressional committees have also expressed concerns
about the transparency by which FTA makes changes to the evaluation and
rating process, as shown by the bills to reauthorize surface
transportation programs--introduced in early 2005--that offer a more
formalized approach for providing notice to and soliciting comment from
project sponsors.[Footnote 28] In particular, the House reauthorization
bill would require that FTA provide notice and an opportunity for
comment at least 60 days before issuing any "nonregulatory substantive
changes."[Footnote 29] The Senate proposal states that FTA should issue
periodic descriptions of the rating criteria and allow for public
comment.[Footnote 30] (See app. IV for information about other changes
proposed by the House and Senate.)
Project sponsors offered a number of suggestions for improving FTA's
communication about changes to the New Starts process. For example,
nine project sponsors told us that FTA should provide them with an
opportunity to offer input into a change before it is made. Thirteen
project sponsors mentioned that FTA should provide them with more lead
time when the agency requires them to implement a change. For example,
a number of project sponsors expressed concern that the recent
technical changes to the cost-effectiveness calculation announced in
April 2005 were made too close to when New Starts information is due--
June for travel forecasts and "make the case" documents and August for
the remaining application materials.[Footnote 31] In addition, eight
project sponsors expressed that FTA should develop a standard schedule
for when it announces and implements changes to the New Starts process.
For example, one project sponsor suggested that FTA announce all
changes at its New Starts roundtable a year before the changes are to
be implemented.
Our previous body of work on organizational transformation[Footnote 32]
indicates that communication with stakeholders should be a top priority
for any agency, is most effective when done early and often in any
process, and is central to forming the partnerships that are needed to
develop and implement an organization's strategies. An effective
communication strategy should facilitate an honest two-way exchange
with, and allow for feedback from, stakeholders. This communication is
central to forming the effective internal and external partnerships
that are vital to a program's success. For the past 2 years, FTA
officials have announced at the roundtables that they are planning to
develop and implement a strategy for improving communication among FTA
offices and between FTA and project sponsors. FTA officials have told
us that the strategy could include providing a single FTA point of
contact for project sponsors and improving the distribution of policy,
guidance, and procedures, perhaps using tools such as a listserve or
webinar, as it did when it sought comments from project sponsors on the
recent cost-effectiveness changes.[Footnote 33] FTA has implemented new
communications tools, such as the webinar, but has not developed a
comprehensive communications strategy.
New Starts Measures Have Evolved Over Time, but Not All Measures Count
Toward a Project's Rating:
Many of the measures FTA uses to evaluate and rate New Starts projects
predate TEA-21 and have evolved over time. In developing these
measures, FTA has historically sought industry input by way of formal
rulemaking as well as outreach sessions, workshops, and reports.
Although both TEA-21 and FTA's current New Starts program regulations
emphasize the importance of using a multiple-measure approach for
evaluating projects, FTA assigns a 50 percent weight to both the cost-
effectiveness and the land use criteria when developing the project
justification summary rating. The other three project justification
criteria are not weighted, although the mobility improvements criterion
is used as a "tie-breaker" when the average of the cost-effectiveness
and land use ratings falls equally between two ratings (e.g., between
"medium" and "medium-high").[Footnote 34] According to FTA officials,
FTA does not use these criteria because the underlying measures have
weaknesses that diminish their use in distinguishing among projects.
Project sponsors we interviewed offered suggestions for improving all
of the project justification measures.
Congress Directed FTA to Use Multiple Criteria in Evaluating and Rating
Projects:
Through the Intermodal Surface Transportation Efficiency Act of 1991
(ISTEA) and TEA-21, Congress has directed FTA to use multiple criteria
in evaluating and rating New Starts projects. In particular, TEA-21
identifies a series of financial and project justification criteria
that reflect a broad range of benefits and effects of the proposed
projects. The financial criteria include the share of non-New Starts
funding and the capital and operating finance plans. The project
justification criteria identified by TEA-21 include mobility
improvements, cost-effectiveness, operating efficiencies, and
environmental benefits. TEA-21 also identifies several additional
statutory "considerations" to the evaluation process, including land
use issues. TEA-21 directs FTA to "evaluate and rate the project as
'highly recommended,' 'recommended,' or 'not recommended,' based on—the
project justification criteria," among other things.[Footnote 35] FTA
has recognized and acknowledged the congressional intent for New Starts
projects to be evaluated using multiple measures. For instance, in a
January 2005 report to Congress, FTA states that it is "clear that
Congress intends FTA to evaluate projects based on more than cost-
effectiveness criteria" and that the "statutory framework is consistent
with the concept that a wide range of benefits should be considered in
evaluating projects."[Footnote 36]
FTA also has endorsed the concept of using multiple criteria to
evaluate and rate New Starts projects and FTA's New Starts regulations
set forth a multiple measure evaluation process. The New Starts
regulations state that "FTA will combine the ratings for each of the
financial rating factors and project justification criteria into
overall 'finance' and 'justification' ratings—. These ratings will then
be combined into the single, overall project ratings."[Footnote 37]
Further, in response to a comment on the cost-effectiveness measure in
the proposed New Starts regulations, FTA stated the following:
"It is important to note that the measure for cost-effectiveness is not
intended to be a single, stand-alone indicator of the merits of a
proposed new starts project. It is but one part of the multiple measure
method that FTA uses to evaluate project justification under the
statutory criteria. While cost-effectiveness is an important
consideration, so are mobility improvements, environmental benefits,
and the other factors described both in TEA-21 and elsewhere in this
rule."[Footnote 38]
FTA has repeated the importance of using a multiple measure approach in
many reports and other documents since then, including its annual
reports to Congress. The New Starts regulations identify the measures
FTA will use to evaluate projects from the project justification
criteria outlined in TEA-21.[Footnote 39] Figure 5 shows the project
justification criteria identified by TEA-21 and the associated measures
identified by FTA in the New Starts regulations. (See app. V for more
detail on these measures.)
Figure 5: Criteria and Measures for Evaluating Projects, as Outlined in
TEA-21 and FTA's New Starts Program Regulations:
[See PDF for image]
[A] Other factors can include environmental justice and equity issues,
economic development initiatives, innovative financing, etc.
[B] TSUB = Transportation System User Benefits.
[End of figure]
FTA Used Industry Input to Develop Measures for All Criteria Identified
in TEA-21:
The measures FTA uses to evaluate and rate New Starts projects have
evolved over time, beginning years before TEA-21, through an iterative
process involving Congress and the transit industry. FTA (then known as
UMTA) introduced the first system for rating New Starts projects in
1984. At that time, projects were rated on cost-effectiveness (cost per
new rider) and local financial commitment. Through ISTEA in 1991,
Congress added mobility improvements, environmental benefits, and
operating efficiencies to the list of criteria FTA should use to
evaluate projects and also added land use policies as an additional
factor for consideration.
FTA circulated a policy paper in 1994, asking for public comment on its
proposed measures and procedures for assessing projects to address the
requirements laid out in ISTEA. The agency received 31 responses from
transit operators, metropolitan planning organizations, state
departments of transportation, and other interested parties. FTA used
these responses in finalizing its criteria and measures in a notice
published in 1996 in the Federal Register. In the 1994 policy paper,
FTA also solicited comments on the appropriateness of using a multiple-
measure method for evaluating projects. Respondents generally agreed
that this was appropriate, although they were split on how (or whether)
the various criteria should be weighted. FTA formally adopted the
multiple-measure approach in 1996.
TEA-21 required FTA to issue regulations for the evaluation and rating
process, and FTA used that rulemaking opportunity to revise several of
the project justification measures. Before and during the rulemaking
process, FTA conducted outreach sessions around the country, soliciting
comments on its processes and procedures for managing the New Starts
program. FTA issued a Notice of Proposed Rulemaking in 1999 and
received comments from 41 individuals and organizations. In response to
the comments, FTA made several changes to the project justification
measures. For example, many of these commenters objected to the "cost
per new rider" measure for cost-effectiveness, saying that the focus on
new riders ignores benefits provided to other riders, which may bias
the measure against cities with "mature" transit systems. In response,
FTA replaced it with the incremental cost per hour of TSUB measure (to
capture benefits to both new and existing riders). The agency also
added a mobility improvements measure for employment near stations to
complement the existing low-income households near stations measure in
response to industry comments that a system that is located near low-
income households is of little use to residents unless it can also
provide access to employment and other activity centers.
Since issuing the regulations, FTA has continued efforts to fine-tune
and improve some of the project justification measures. For example,
FTA convened a panel of experts from the Urban Land Institute to
discuss the land use measures and is seeking industry input on possible
changes. FTA's Strategic Business Plan for Fiscal Year 2005 includes
deliverables to (1) develop a land use measure that is more
quantifiable, offers a better basis for distinguishing among projects,
and provides grantees information with which to improve their projects
and (2) develop a methodology for measuring the congestion relief
benefits of New Start projects.[Footnote 40] Both are scheduled to be
completed by summer 2005. FTA officials told us that the fiscal year
2005 program plan included research funds for these efforts. In
addition, as discussed previously, FTA recently instituted technical
modifications to the cost-effectiveness measure.
FTA Does Not Use All Project Justification Criteria in Determining
Ratings Because of Perceived Weaknesses in the Measures:
Despite the requirement to use multiple measures, FTA does not use
three of the five project justification criteria in calculating a
project's rating. Specifically, FTA currently assigns a weight of 50
percent each to the cost-effectiveness and land use criteria; the other
three project justification criteria are not assigned weights (see fig.
6). In its annual report for fiscal year 2006, FTA stated that it
assigns individual ratings for the other three project justification
criteria and reports them in the annual report, but these ratings "are
not considered in the determination of an overall project justification
rating."[Footnote 41] FTA does, however, use all three financial
commitment criteria in developing a project's rating.
Figure 6: Weights Used to Determine Project Justification and Financial
Summary Ratings for New Starts Projects:
[See PDF for image]
Note: Numbers do not add to 100 due to rounding.
[End of figure]
FTA officials told us that they do not use the mobility improvements,
environmental benefits, and operating efficiencies criteria in
determining the project justification summary rating because the
measures do not, as currently structured, provide meaningful
distinctions among competing New Starts projects. Many project sponsors
we interviewed had similar views, noting that individual projects are
too small to have much impact on the whole region or the whole transit
system. For example, one of the environmental benefits measures
requires project sponsors to measure the project's impact on the annual
number of tons of emissions forecast for the region for various
pollutants. FTA officials also said that they do not assign weight to
the mobility improvements measures in determining the project
justification rating because they believe that the employment and low-
income household measures do not meaningfully distinguish among
projects, and the user benefits are already captured in the cost-
effectiveness measure. Therefore, to count the user benefits as part of
the mobility improvements would result in a double-counting of the
benefits.[Footnote 42] In contrast, FTA officials told us that the cost-
effectiveness and land use measures help to make meaningful
distinctions among projects. For example, according to FTA, existing
transit supportive plans and policies demonstrate an area's commitment
to transit projects and are a strong indicator of a project's future
success.
Because FTA officials believe that these project justification measures
do not provide meaningful distinctions among projects and are not
factored into a project's rating, FTA does not stringently evaluate the
projects on the associated criteria. For instance, according to FTA's
New Starts reporting instructions, every project must submit data on
operating efficiencies and automatically receives a score of medium on
that criterion. Similarly, a project that is in a nonattainment area
for a pollutant and that demonstrates a projected decrease in that
pollutant gets a high environmental benefits rating, while a project in
an attainment area that demonstrates a decrease gets a medium
environmental benefits rating.[Footnote 43]
The New Starts regulations do not specify the weights that should be
assigned to each project justification criterion. FTA sought industry
input on whether there should be a weighting system in the last
rulemaking process; however, according to FTA officials, the agency did
not receive input that was useful to inform its policy. Consequently,
the regulations are silent on the weights that should be assigned to
each criterion and do not prohibit FTA from treating various criteria
as more important than other criteria. FTA instituted its current
weighting system with the fiscal year 2004 evaluation cycle after
determining that the operating efficiencies, environmental benefits,
and mobility improvements measures do not effectively distinguish among
projects and that there is overlap among the mobility and cost-
effectiveness measures. Because the regulations do not set forth the
weights that should be assigned to each criterion, FTA did not amend
the regulations when it instituted the current weighting system.
Although FTA does not use all of the project justification criteria
identified in TEA-21 to calculate project ratings, project sponsors
must submit information for all five project justification criteria and
FTA publishes this information for each project in the New Starts
annual report. FTA officials offered several reasons for requiring
project sponsors to continue to provide this information. First, TEA-21
requires FTA to consider these criteria when evaluating projects. Even
though not all of the project justification criteria count toward the
rating, FTA officials told us that they review and consider all five
criteria and, therefore, are operating within the evaluation framework
established in TEA-21 and the New Starts regulations. Second, FTA
officials said that they believe that mobility improvements,
environmental benefits, and operating efficiencies are important and
should be a part of the evaluation process, even though FTA does not
yet have measures for these criteria that help make distinctions among
projects for the purpose of rating and funding decisions. No measures
are specified in TEA-21, which only describes the criteria FTA should
use to evaluate and rate projects, so FTA has the flexibility to revise
the measures as needed. FTA officials stated that they continue to
examine and pursue options to improve the measures and that FTA has
committed approximately $500,000 of its fiscal year 2005 research funds
to continue its work to improve the New Starts measures. The officials
also said that they would wait to change these measures until
legislation governing the New Starts program is reauthorized. They said
that they will have to institute a formal rulemaking process at that
time and would use that opportunity to solicit public comment on the
New Starts evaluation and rating process. Third, FTA officials told us
that information on these three criteria is presented in the annual
report and may be useful to Congress and local decisionmakers. A number
of project sponsors we spoke to expressed frustration that they must
prepare and submit information for all the measures for the five
project justification criteria even though some of this information
does not contribute to the projects' ratings.
Project Sponsors and Others Identified Strengths and Weaknesses of the
Measures and Suggested Improvements:
Project sponsors we interviewed, as well as FTA, the Department of
Transportation's Inspector General, and other industry experts have
identified various strengths and weaknesses of the project
justification measures used to evaluate the New Starts projects. For
example, all of these sources acknowledge that FTA's measure of cost-
effectiveness does not capture benefits that accrue to highway users as
more people switch to the improved transit system and highway
congestion decreases. According to the department's Inspector General,
the omission of highway travel time savings means that the benefits
from proposed projects that convey significant travel time savings for
motorists are not recognized in the selection process. FTA noted that
current local models used to estimate future travel demand for New
Starts are incapable of estimating reliable highway travel time savings
as a result of the New Starts project. According to the department's
Inspector General, this limitation is due to unreliable local data on
highway speeds.[Footnote 44] FTA is working with the Federal Highway
Administration to study ways to remedy this problem. Table 3 shows the
strengths, weaknesses, and other concerns most commonly mentioned by
the New Starts project sponsors we interviewed.
Table 3: Strengths, Weaknesses, and Other Concerns about New Starts
Measures, as Identified by Project Sponsors:
Criterion: Mobility improvements;
Measure: Transportation System User Benefits (TSUB) per project
passenger mile;
Strengths:
* Captures benefits to new and existing riders (3); Weaknesses:
* Does not include highway benefits (4);
* Favors longer projects (3);
Other concerns[A]:
* Unclear what it means or how it is used (6);
* Difficult to explain (5);
* Difficult/time-consuming to calculate (4).
Criterion: Mobility improvements;
Measure: Employment near stations;
Strengths:
* Measures the potential transit market (4); Weaknesses:
* Does not include projected or planned employment (3);
* Does not capture benefits to the whole system or corridor (3).
Criterion: Mobility improvements;
Measure: Low-income households near stations;
Strengths:
* Measures the potential transit market (7);
* Measures urban revitalization, equity, or social justice (3);
Weaknesses:
* Does not measure "choice" riders (e.g., commuters); does not reflect
purpose of all transit systems (8);
* Does not really measure mobility (5).
Criterion: Environmental benefits;
Measure: Change in regional pollutant emissions;
Strengths:
* Directly addresses pollution reduction (4); Weaknesses:
* Individual project has a small impact on the region (13).
Criterion: Environmental benefits;
Measure: Change in regional energy consumption;
Strengths: No consensus on strengths; Weaknesses:
* Individual project has a small impact on the region (10);
* This is not within the transit agency's control (3).
Criterion: Environmental benefits;
Measure: Environmental Protection Agency's air quality designation;
Strengths:
* Helps determine where transit is most needed (3); Weaknesses:
* Individual project has a small or no impact on the region's
designation (6);
* This is not within the transit agency's control (6);
Other concerns[A]:
* Designation should not matter--all areas should be trying to improve
air quality (3).
Criterion: Operating efficiencies;
Measure: System operating cost per passenger mile;
Strengths:
* Understandable, simple, straightforward, reasonable (6); Weaknesses:
* Individual project has small impact on systemwide efficiency (5);
* Incomplete by itself; needs context (3).
Criterion: Cost-effectiveness;
Measure: Incremental cost per hour of TSUB;
Strengths:
* Captures/quantifies project benefits in one number for comparison
(9);
* Reasonable, understandable (3);
* Improvement over old measure (3); Weaknesses:
* Sensitive to variations in underlying assumptions (5);
* Does not include highway benefits (5);
* Does not account for other local goals or benefits (5);
* Does not include project's impact on land use[B] (3);
Other concerns[A]:
* Complicated to calculate and explain (7);
* Thresholds/caps seem arbitrary (4).
Criterion: Land use;
Measure: Existing land use;
Strengths:
* Reflects current or future ridership (5);
* Demonstrates project's benefits/need (3); Weaknesses:
* Focuses on present, but project is built for the future (5);
* Subjective (3).
Criterion: Land use;
Measure: Transit supportive plans and policies;
Strengths:
* Emphasizes link between land use and transit (8); Weaknesses:
* Qualitative, difficult to document/measure (5).
Criterion: Land use;
Measure: Performance and impact of policies[C];
Strengths:
* Measures actual public local support for transit (3);
* Shows opportunity for expansion/development (3); Weaknesses:
* Qualitative, subjective, vague (8).
Source: GAO.
Note: Numbers in parentheses indicate response frequency. This table
only includes responses that were mentioned by at least 3 of the 26
project sponsors we interviewed.
[A] Project sponsors raised these other concerns during our discussions
on the measures' strengths and weaknesses.
[B] For example, one project sponsor said that the measure does not
account for local land use policies and the resulting development
potential in the area, such as the impact of that city's 20-year zoning
plans.
[C] This measure includes performance of land use policies and
potential impact of transit project on regional land use.
[End of table]
In addition, in a recent report on the benefits and costs of
transportation improvements, we identified challenges in measuring the
benefits and costs of transit investments--some of which are relevant
to the measures used by FTA to evaluate New Starts projects.[Footnote
45] For example, desirable changes in land use are indirect benefits of
a transportation investment, which are difficult to forecast and
quantify. We also reported that social benefits such as reductions in
environmental costs--including reduced emissions--were difficult to
quantify and value. Additionally, we reported that there is great
variation in the models local transportation planning agencies use to
develop travel forecasts (which underlie many of the New Starts
measures), producing significant variation in forecast quality and
limiting the ability to assess quality against the general state of
practice. Some experts have also found that travel demand models tend
to predict unreasonably bad conditions in the absence of a proposed
highway or transit investment. In particular, travel forecasting does
not contend well with land-use changes or effects on nearby roads or
other transportation alternatives that result from transportation
improvements or growing congestion. Before conditions get as bad as
they are forecasted, people make other changes, such as residence or
employment changes, to avoid the excessive travel costs.[Footnote 46]
Our previous work has stated that agencies successful in measuring
performance had performance measures that, among other things,
demonstrate results and cover multiple priorities. Specifically,
successful measures (1) are aligned with agencywide goals and mission
and are clearly communicated throughout the organization; (2) are
clearly stated, with a name and definition that are consistent with the
methodology used to calculate the measures; (3) have a measurable
target; (4) are objective; (5) are reliable; (6) cover core program
activities; (7) have limited overlap with other measures; (8) provide
balance in ensuring that various priorities are covered; and (9)
address governmentwide priorities, such as quality and cost of service.
For example, measures that are not objective may result in performance
assessments that are systematically over-or understated, and a lack of
balance could create skewed incentives when measures overemphasize some
goals.[Footnote 47] While these successful attributes were developed
specifically for performance measures, they also could be useful in
determining how to improve other types of measures, such as those FTA
uses to evaluate and rate New Starts projects.
The sponsors of the 26 projects we interviewed had many suggestions for
improving the project justification measures. These suggestions ranged
from adding measures to the mobility improvements, environmental
benefits, and land use criteria to changing the way in which operating
efficiencies are calculated. The most commonly cited suggestions are
listed in table 4, but we did not determine whether the suggestions
were appropriate or feasible. FTA officials told us that they received
limited suggestions for specific measures or methodologies for mobility
improvements, environmental benefits, and operating efficiencies during
the rulemaking process.
Table 4: Suggestions for Improving New Starts Measures, as Identified
by Project Sponsors:
Area: Mobility improvements;
Suggestions:
* Add measures, such as population density, residential density, number
of high-income households served, projected or planned employment near
stations, retail and stadiums near stations, transit-oriented
development, accessibility, or number of new riders (10).
Area: Environmental benefits;
Suggestions:
* Add measures or replace existing measures with movement toward a
sustainable transit system, effect of project on automobile congestion,
number of cars taken off the road, noise quality, preservation of open
space, or other measures identified in the Environmental Impact
Statement (4).
Area: Operating efficiencies;
Suggestions:
* Measure operating cost per passenger or per passenger hour instead of
per passenger mile (3).
Area: Cost-effectiveness;
Suggestions:
* Add measures, such as cost per new rider, in addition to the current
measure of incremental cost per hour of Transportation System User
Benefits (5);
* Adjust thresholds for inflation[A] (3).
Area: Land use;
Suggestions:
* No consensus on suggestions.
Source: GAO.
Notes: Numbers in parentheses indicate response frequency. This table
only includes responses that were mentioned by at least 3 of the 26
project sponsors we interviewed.
[A] FTA announced it was adjusting the cost-effectiveness thresholds
for inflation on April 29, 2005, after we had completed our interviews.
[End of table]
Conclusions:
TEA-21 and FTA's New Starts regulations clearly state that New Starts
projects should be evaluated and rated using multiple criteria.
Further, the statute and regulations identify the project justification
criteria that should be used in the evaluation process, including
mobility improvements, environmental benefits, operating efficiencies,
cost-effectiveness, and land use, as well as the financial commitment
criteria that should be used in the evaluation process. The regulations
also identify the measures that will be used to operationalize the
criteria. Although FTA uses all three financial criteria, in practice,
FTA uses only two of the project justification criteria--cost-
effectiveness and land use--to calculate a project's overall rating.
FTA's reliance on two of the five project justification criteria,
coupled with the recent cost-effectiveness practice for funding--which
further emphasizes one criterion--is drifting away from the multiple-
measure evaluation and rating process outlined in TEA-21 and the
current New Starts regulations. According to FTA officials, FTA does
not assign weights to environmental benefits, operating efficiencies,
and mobility improvements because of weaknesses in the measures and the
overlap of some measures that could result in double-counting of
benefits. FTA has made notable progress in improving the measures for
cost-effectiveness and land use, including seeking advice from experts
and the transit industry and conducting pilot tests since the enactment
of TEA-21. However, given that FTA has been statutorily directed to
also use environmental benefits, operating efficiencies, and mobility
improvements in evaluating and rating projects, it is imperative that
FTA either pay additional attention to improving these three criteria
so that they can be more explicitly used in the evaluation process or
explicitly demonstrate the linkages between these criteria and the
measures used.
FTA has made a number of changes intended to enhance the rigor of the
program over the past 6 years, and some of these changes, such as the
Before and After study, risk assessment, and cost-effectiveness
practice could help FTA hold project sponsors accountable for results
and maximize the benefits of each dollar invested. However, FTA could
improve the transparency of changes made to the New Starts program by
giving project sponsors an opportunity to review and comment on any
substantive changes before they are implemented. The Freedom of
Information and Administrative Procedure Acts establish formal
processes for notifying the public and making changes to federal
programs, including soliciting comments about proposed changes. If FTA
officials determine that a formal rulemaking process is unnecessary,
FTA could still provide project sponsors an opportunity to review and
comment on any substantive changes proposed for the New Starts program,
potentially avoiding some of the implementation problems that have
occurred in the past. In addition, the review and comment period could
help FTA improve the proposed changes as well as gain industry buy-in
and support for changes--elements that are critical for success.
FTA could also strengthen its communication efforts by improving its
Web site so that project sponsors view it as a viable source for
obtaining information about changes to the New Starts program. The Web
site could provide a central forum for comprehensive, up-to-date
information on the New Starts program and could also be useful for
publicizing FTA's activities to improve the application, evaluation,
rating, and oversight processes. Much of this information is already
available on FTA's Web site. However, the information that remains is
scattered in different locations and many project sponsors told us that
it was difficult to locate needed information. Making the information
easier to find could help reduce confusion about the New Starts
process.
Recommendations for Executive Action:
To ensure that the New Starts regulations reflect FTA's current
evaluation and rating process, and to ensure that FTA's New Starts
evaluation process and policies are objective, transparent, and follow
the intent of federal statute, we recommend that the Secretary of
Transportation direct the Administrator, FTA, to take the following
four actions:
* ensure that the agency's regulations governing the New Starts
evaluation and rating process reflect FTA's current weighting practices
for the criteria when the regulations are revised;
* improve the measures for evaluating New Starts projects so that all
five project justification criteria can be used in determining a
project's overall rating, or provide a crosswalk in the agency's New
Starts regulations showing clear linkages between the criteria outlined
in statute and the criteria and measures used in the rating process;
* publish future changes to the New Starts program in the Federal
Register and subject future changes to the New Starts program to the
rulemaking process or, at a minimum, a 30-day informal review and
comment period, as appropriate. As part of this process, the agency
should develop criteria for determining which changes should be subject
to the rulemaking process, as outlined in federal statute, or to an
informal review and comment period. At a minimum, these criteria could
include changes that impose new reporting requirements or new analysis
on project sponsors, changes to the principles used to recommend
projects for funding, and changes to the weights assigned to the
criteria used to evaluate and rate projects. The criteria should be
communicated to Congress and to project sponsors and others in the
transit community; and:
* consolidate information and guidance related to the New Starts
program in one location on the agency's Web site and regularly review
this information to ensure it is up to date and easy to access.
Agency Comments and Our Evaluation:
We provided a draft of this report to the Department of Transportation
for review and comment. Officials from the Department and FTA indicated
that they generally agreed with the report's findings, conclusions, and
recommendations. FTA officials also provided technical clarifications,
which we incorporated as appropriate.
We are sending copies of this report to congressional committees with
responsibilities for transit issues; the Secretary of Transportation;
the Administrator, Federal Transit Administration; and the Director,
Office of Management and Budget. We also will make copies available to
others upon request. In addition, this report will be available at no
charge on GAO's Web site at [Hyperlink, http://www.gao.gov].
If you or your staff have any questions on matters discussed in this
report, please contact me at [Hyperlink, siggerudk@gao.gov]. GAO
contacts and key contributors to this report are listed in appendix VI.
Signed by:
Katherine Siggerud:
Director, Physical Infrastructure:
[End of section]
Appendixes:
Appendix I: Scope and Methodology:
To address our objectives, we reviewed the administration's fiscal year
2006 budget request, the Federal Transit Administration's (FTA) annual
New Starts report, FTA's New Starts regulations, FTA's 2004 and 2005
reporting instructions for the New Starts program, federal statutes
pertaining to the new Starts program, and previous GAO reports. We also
interviewed FTA officials and representatives from the American Public
Transportation Association, New Starts Working Group, and the
Association of Metropolitan Planning Organizations. In addition, we
attended FTA's New Starts roundtables with project sponsors in New York
and San Francisco in April and May 2005, respectively.
We also conducted semistructured interviews with project sponsors from
the 26 projects that were evaluated and rated in the fiscal year 2006
evaluation cycle (see table 5). These interviews were designed to gain
project sponsors' perspectives on a number of topics, including the
manner in which FTA communicates changes to the New Starts application,
evaluation, rating, and project development oversight processes; the
impact of the changes that FTA has made to the application, evaluation,
rating, and project development oversight processes since the fiscal
year 2001 evaluation cycle--the first full evaluation and rating cycle
after the enactment of the Transportation Equity Act for the 21ST
Century (TEA-21); and the measures FTA uses to evaluate projects. To
verify the clarity, length of time of administration, and
understandability of the interview questions, as well as to determine
whether respondents had sufficient knowledge and information to answer
the questions, we pretested the questions with three project sponsors.
We made changes to the content and format of the final set of interview
questions as a result of these pretests. After conducting the
interviews with sponsors from all 26 projects, we used a content
analysis to systematically determine project sponsors' views on key
interview questions and identify common themes in project sponsors'
responses. Two analysts reached consensus on the coding of the
responses, and a third reviewer was consulted in case of disagreements,
to ensure that the codes were reliable.
Table 5: Projects Contacted for Our Review:
Final design or proposed for full funding grant agreement:
Location: Charlotte;
Project: South Corridor LRT.
Location: Las Vegas;
Project: Resort Corridor Downtown Extension.
Location: New York City;
Project: Long Island Rail Road East Side Access.
Location: Phoenix;
Project: Central Phoenix/East Valley LRT.
Location: Pittsburgh;
Project: North Shore LRT Connector.
Location: Raleigh-Durham;
Project: Regional Rail System.
Location: Washington County;
Project: Wilsonville to Beaverton Commuter Rail.
Preliminary engineering:
Location: Boston;
Project: Silver Line Phase III.
Location: Dallas;
Project: Northwest/Southeast Light Rail MOS.
Location: Denver;
Project: West Corridor LRT.
Location: Fort Collins;
Project: Mason Transportation Corridor.
Location: Hartford;
Project: New Britain-Hartford Busway.
Location: Los Angeles;
Project: Mid-City/Exposition Corridor LRT.
Location: Miami;
Project: North Corridor Metrorail Extension.
Location: Minneapolis-Big Lake;
Project: Northstar Corridor Rail.
Location: New Orleans;
Project: Desire Streetcar Line.
Location: New York City;
Project: Second Ave. Subway MOS.
Location: Norfolk;
Project: Norfolk LRT.
Location: Northern VA;
Project: Dulles Corridor Metrorail Extension to Wiehle Ave.
Location: Orange County;
Project: Centerline LRT.
Location: Philadelphia;
Project: Schuylkill Valley Metrorail.
Location: Portland;
Project: South Corridor I-205/Portland Mall LRT.
Location: Salt Lake City;
Project: Weber County to Salt Lake City Commuter Rail.
Location: San Diego;
Project: Mid-Coast LRT Extension.
Location: San Francisco;
Project: New Central Subway.
Location: Santa Clara County;
Project: Silicon Valley Rapid Transit Corridor.
Sources: GAO and FTA.
Note: LRT = Light Rail Transit; MOS = Minimum Operable Segment:
[End of table]
To ensure the reliability of information presented in this report, we
interviewed FTA officials about FTA's policies and procedures for
compiling the New Starts annual reports, including FTA's data
collection and verification practices for New Starts information.
Specifically, we asked them whether their policies and procedures had
changed significantly since we reviewed them for our 2004 report on New
Starts.[Footnote 48] FTA officials told us that there were no
significant changes in their data collection and verification policies
and procedures for New Starts information. Therefore, we concluded that
the FTA information presented is sufficiently reliable for the purposes
of this report.
We conducted our work from November 2004 through May 2005 in accordance
with generally accepted government auditing standards, including
standards for data reliability.
[End of section]
Appendix II: Projects with Existing FFGAs and Projects in Preliminary
Engineering and Final Design in the Fiscal Year 2006 Cycle:
Dollars in millions.
Projects with existing full funding grant agreements (FFGAs):
CA;
Location/project: Los Angeles--Metro Gold Line East Side Extension;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $80.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $490.7.
CA;
Location/project: San Diego--Mission Valley East Light Rail Transit
(LRT) Extension;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $7.7;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $330.0.
CA;
Location/project: San Diego--Oceanside-Escondido Rail Corridor;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $12.2;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $152.1.
CA;
Location/project: San Francisco--Bay Area Rapid Transit Extension to
San Francisco Airport;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $81.9;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $750.0.
CO;
Location/project: Denver--Southeast Corridor LRT;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $80.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $525.0.
IL;
Location/project: Chicago--Douglas Branch Reconstruction;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $45.2;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $320.1.
IL;
Location/project: Chicago--North Central Corridor Commuter Rail;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $20.6;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $135.3.
IL;
Location/project: Chicago--Ravenswood Line Extension;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $40.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $245.5.
IL;
Location/project: Chicago--South West Corridor Commuter Rail;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $7.3;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $103.0.
IL;
Location/project: Chicago--Union Pacific West Line Extension;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $14.3;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $80.8.
MD;
Location/project: Baltimore--Central LRT Double-Track;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $12.4;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $120.0.
NJ;
Location/project: Northern New Jersey--Hudson-Bergen Minimum Operable
Segment (MOS)-2;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $100.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $500.0.
OH;
Location/project: Cleveland--Euclid Corridor Transportation Project;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $24.8;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $82.2.
OR;
Location/project: Portland--Interstate MAX LRT Extension;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $18.1;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $257.5.
PR;
Location/project: San Juan--Tren Urbano;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $10.2;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $307.4.
WA;
Location/project: Seattle--Central Link Initial Segment;
Overall project rating/status: FFGA;
Fiscal year 2006 recommended funding: $80.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $500.0.
Subtotal;
Fiscal year 2006 recommended funding: $634.6;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $4,899.6.
Projects recommended for FFGAs:
AZ;
Location/project: Phoenix--Central Phoenix/East Valley LRT;
Overall project rating/status: Recommended;
Fiscal year 2006 recommended funding: $90.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $587.2.
NC;
Location/project: Charlotte--South Corridor LRT;
Overall project rating/status: Recommended;
Fiscal year 2006 recommended funding: $55.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $192.9.
NY;
Location/project: New York--Long Island Rail Road East Side Access;
Overall project rating/status: Highly recommended;
Fiscal year 2006 recommended funding: $390.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $2,632.0.
PA;
Location/project: Pittsburgh--North Shore LRT Connector;
Overall project rating/status: Recommended;
Fiscal year 2006 recommended funding: $55.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $217.7.
Subtotal;
Fiscal year 2006 recommended funding: $590.0;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $3,629.8.
Projects in final design:
MO;
Location/project: Kansas City--Southtown Bus Rapid Transit (BRT);
Overall project rating/status: Exempt;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $12.3.
NC;
Location/project: Raleigh-Durham--Regional Rail System;
Overall project rating/status: Not rated;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $416.1.
NV;
Location/project: Las Vegas--Resort Corridor Downtown Extension;
Overall project rating/status: Not recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $159.7.
OR;
Location/project: Washington County--Wilsonville to Beaverton Commuter
Rail[A];
Overall project rating/status: Recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $51.8.
TN;
Location/project: Nashville--East Corridor Commuter Rail;
Overall project rating/status: Exempt;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $23.5.
Subtotal;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $663.4.
Projects in preliminary engineering:
AK;
Location/project: Wasilla--South Wasilla Track Realignment;
Overall project rating/status: Exempt;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: N/A.
CA;
Location/project: Los Angeles--Exposition Corridor LRT;
Overall project rating/status: Not rated;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $276.0.
CA;
Location/project: Orange County--CenterLine LRT;
Overall project rating/status: Not rated;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $482.8.
CA;
Location/project: San Diego--Mid-Coast LRT Extension[A];
Overall project rating/status: Recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $65.8.
CA;
Location/project: San Francisco--Central Subway;
Overall project rating/status: Recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $762.2.
Projects in preliminary engineering:
CA;
Location/project: Santa Clara County--Silicon Valley Rapid Transit
Corridor;
Overall project rating/status: Not recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $973.0.
CO;
Location/project: Denver--West Corridor LRT[A];
Overall project rating/status: Recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $249.0.
CO;
Location/project: Fort Collins--Mason Transportation Corridor;
Overall project rating/status: Not recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $33.0.
CT;
Location/project: Hartford--New Britain-Hartford Busway;
Overall project rating/status: Not recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $168.5.
DE;
Location/project: Wilmington--Wilmington to Newark Commuter Rail
Improvements;
Overall project rating/status: Exempt;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $24.9.
FL;
Location/project: Miami--North Corridor Metrorail Extension;
Overall project rating/status: Not rated;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $421.3.
FL;
Location/project: Tampa Bay--Tampa Bay Regional Rail[B];
Overall project rating/status: Not recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $727.7.
LA;
Location/project: New Orleans--Desire Streetcar Line;
Overall project rating/status: Not recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $68.7.
MA;
Location/project: Boston--Silver Line Phase III;
Overall project rating/status: Recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $468.3.
MN;
Location/project: Minneapolis-Big Lake--Northstar Corridor Rail;
Overall project rating/status: Not recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $132.5.
NY;
Location/project: New York--Second Avenue Subway MOS[A];
Overall project rating/status: Highly recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $1,300.0.
OR;
Location/project: Portland--South Corridor I-205/Portland Mall LRT;
Overall project rating/status: Recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $296.2.
PA;
Location/project: Harrisburg--CORRIDORone Rail MOS;
Overall project rating/status: Exempt;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $24.9.
PA;
Location/project: Philadelphia--Schuylkill Valley MetroRail;
Overall project rating/status: Not recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $2,071.1.
RI;
Location/project: Providence--South County Commuter Rail;
Overall project rating/status: Exempt;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $24.9.
TX;
Location/project: Dallas--Northwest/Southeast Light Rail MOS[A];
Overall project rating/status: Recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $700.0.
TX;
Location/project: El Paso--Sun Metro Area Rapid Transit Starter Line;
Overall project rating/status: Exempt;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: N/A.
UT;
Location/project: Salt Lake City--Weber County to Salt Lake City
Commuter Rail[A];
Overall project rating/status: Recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $466.8.
VA;
Location/project: Norfolk--Norfolk LRT;
Overall project rating/status: Not rated;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $100.7.
VA;
Location/project: Northern Virginia--Dulles Corridor Metrorail Project-
Extension to Wiehle Avenue;
Overall project rating/status: Recommended;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $760.5.
Subtotal;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $10,598.8.
Total;
Fiscal year 2006 recommended funding: $1,224.6;
Total New Starts funding scheduled by FFGA or requested by project
sponsors: $19,791.6.
Source: FTA.
Note: Totals may not add due to rounding.
[A] Projects proposed for potential federal funding commitments outside
of full funding grant agreements.
[B] This project has since withdrawn from the New Starts application
process.
[End of table]
[End of section]
Appendix III: Status of Previous GAO Recommendations for Improving the
New Starts Evaluation Process:
Report: GAO/T-RCED-00-104;
Recommendation: The Federal Transit Administration (FTA) should
prioritize the projects it rates as highly recommended or recommended
and ready to receive New Starts funds;
Status: FTA officials told us that their recently implemented cost-
effectiveness practice was partly in response to this recommendation.
FTA will generally not recommend funding for a project that does not
achieve at least a "medium" cost-effectiveness rating.
Report: GAO-01-987;
Recommendation: FTA should make commitment authority allocated to
projects for which the federal funding commitments have been withdrawn
available for all projects competing for New Starts funding (i.e.,
"release" the funding);
Status: In fiscal year 2003, FTA released $157 million in commitment
authority reserved for a Los Angeles Mid-City subway project that had
been suspended for more than 3 years.
Report: GAO-03-701;
Recommendation: FTA should amend its regulations governing the level of
federal funding share for projects to reflect its policy of favoring
projects that request less than 60 percent New Starts funding;
Status: FTA disagreed with this recommendation, arguing that its
preference policy is not a legally binding requirement and therefore
should not be reflected in the New Starts regulations. However, the
agency did change the way it characterized the policy in its fiscal
year 2006 New Starts annual report. This document states that FTA
"generally" will not recommend funding for projects that request more
than a 60 percent New Starts share of funding.
Report: GAO-03-701;
Recommendation: FTA should issue additional guidance to transit
agencies describing FTA's expectations regarding the local travel
forecasting models and the specific type of data FTA requires to
calculate the Transportation System User Benefits measure;
Status: FTA completed several actions and has other ongoing efforts to
address the recommendation. (1) FTA provided additional guidance in its
reporting instructions, clarifying the practices that must be followed
in preparing the Transportation System User Benefits measure. (2) FTA
provided additional guidance in July 2003, on how it would evaluate the
measures in the fiscal year 2005 rating process. (3) FTA held several
special workshops for transit agencies on Transportation System User
Benefits, travel forecasting, and development of transit alternatives.
(4) FTA initiated a proactive outreach to project sponsors to help them
prepare and evaluate the measure, identifying weaknesses in their
proposed measure and suggesting improvements. (5) FTA has a number of
ongoing planned improvements to its guidance, including developing a
users guide to the Summit software, developing case studies and
exemplary practices, and updating its guidance on travel forecasting
for New Starts projects.
Report: GAO-04-748;
Recommendation: FTA should clearly explain the basis on which it
decides which projects will be recommended for funding outside of full
funding grant agreements, and what projects must do to qualify for such
a recommendation. These explanations should be included in FTA's annual
New Starts report and other published New Starts guidance;
Status: In its fiscal year 2006 New Starts report, FTA described its
criteria for recommending projects for funding outside of full funding
grant agreements. These criteria include projects that were in or
nearing final design, received overall highly recommended or
recommended ratings, and had cost-effectiveness ratings above a "low."
Report: GAO-04-748;
Recommendation: FTA should examine the impact of its preference policy
on projects currently in the evaluation process, as well as projects in
the early planning stages, and examine whether its policy results in
maximizing New Starts funds and local participation;
Status: FTA has not implemented this recommendation.
Source: GAO.
[End of table]
[End of section]
Appendix IV: Key New Starts Provisions Contained in House and Senate
Reauthorization Bills:
Provision: Streamline the New Starts evaluation process for projects
under $75 million;
Senate proposal:
* Allows the Secretary of Transportation discretion to develop a
streamlined evaluation process for projects requesting less than $75
million in New Starts funds;
* Eliminates the "exempt" classification for projects requesting less
than $25 million in New Starts funding and allows FTA to analyze and
rate all projects through a streamlined process; House proposal:
* Establishes a "Small Starts" program for projects requesting between
$25 million and $75 million in New Starts funding, with a total project
cost of less than $200 million. These projects would be evaluated
through a streamlined rating process;
* Maintains the "exempt" classification for projects requesting less
than $25 million in New Starts funding.
Provision: Expand the definition of eligible projects;
Senate proposal:
* Allows nonfixed-guideway transit projects (e.g., bus rapid transit
operating in nonexclusive lanes) requesting less than $75 million to be
eligible for New Starts funding; House proposal:
* Expands New Starts funding eligibility to include nonfixed-guideway
projects with a majority of fixed-guideway components seeking between
$25 million and $75 million as part of its "Small Starts" initiative.
Provision: Change the rating categories;
Senate proposal:
* Revises the current rating system (that uses "highly recommended,"
"recommended," and "not recommended" ratings) to five levels of
ratings: "high," "medium-high," "medium," "medium-low," and "low.";
House proposal:
* Maintains the current rating system.
Provision: Maintain a maximum federal New Starts share at 80 percent;
Senate proposal:
* Maintains the maximum New Starts share at 80 percent for individual
projects (in contrast to the administration's reauthorization proposal,
which would lower the maximum New Starts share to 50 percent); House
proposal:
* Maintains the maximum New Starts share at 80 percent for individual
projects;
* Specifically prohibits FTA from requiring a nonfederal share that is
more than 20 percent of the project's cost.
Provision: Formalize communication about changes;
Senate proposal:
* Requires FTA to issue periodic descriptions of the review and
evaluation process and criteria and allow for public comment; House
proposal:
* Requires FTA to provide notice and an opportunity for comment at
least 60 days before issuing any nonregulatory substantive changes.
Sources: H.R. 3, 109TH Cong. (2005), and 151 Cong. Rec. S5667-S5669
(daily ed. May 20, 2005).
[End of table]
[End of section]
Appendix V: FTA's Project Justification Measures for Evaluating and
Rating New Starts Projects:
Criterion: Mobility improvements;
Measure: Transportation System User Benefits per project passenger mile
(normalized travel time savings);
Definition: Annual Transportation User Benefits (user expenditure
savings between New Starts baseline and build alternatives) per
forecasted project passenger mile, divided by total Transportation
System User Benefits per passenger mile.
Criterion: Mobility improvements;
Measure: Low-income households served;
Definition: Estimated number of low-income households (i.e., households
below the poverty level) located within 1/2 mile of boarding points
(transit stations) on the proposed New Starts project.
Criterion: Mobility improvements;
Measure: Employment near stations;
Definition: Number of jobs within 1/2 mile of the New Starts project's
proposed transit stations.
Criterion: Environmental benefits;
Measure: Change in criteria pollutant/precursor emissions;
Definition: Annual number of tons of emissions forecast for the region--
comparing conditions under the New Starts investment to the New Starts
baseline alternative--for carbon monoxide, particulate matter, nitrogen
oxides, volatile organic compounds, and carbon dioxide.
Criterion: Environmental benefits;
Measure: Change in regional energy consumption;
Definition: Net impact on energy savings as a result of changes in
automobile and commercial travel in the region, offset in part by the
energy requirements for operation of the proposed transit investment,
measured in British Thermal Units.
Criterion: Environmental benefits;
Measure: Current Environmental Protection Agency regional air quality
designation;
Definition: U.S. Environmental Protection Agency's current air quality
designation for the region, reflecting current compliance with the
National Ambient Air Quality Standards, reported as attainment,
nonattainment, or maintenance for transportation-related pollutants
including ozone, carbon monoxide, particulate matter, and nitrogen
oxides.
Criterion: Operating efficiencies;
Measure: Operating cost per passenger mile;
Definition: Change in systemwide operating cost per passenger mile in
the forecast year, comparing the New Starts build alternative to the
baseline alternative.
Criterion: Cost-effectiveness;
Measure: Incremental cost of Transportation System User Benefits;
Definition: Annualized capital and operating costs divided by
Transportation System User Benefits (annual user expenditure savings)
of the New Starts project as compared to the baseline.
Criterion: Land use;
Measure: Existing land use;
Definition: Includes corridor and station area development, character
(i.e., residential, commercial, mixed-use), pedestrian facilities, and
parking supply.
Criterion: Land use;
Measure: Transit supportive plans and policies;
Definition: Includes growth management, transit supportive corridor
policies, supportive zoning regulations near transit stations, and
tools to implement land use policies.
Criterion: Land use;
Measure: Performance and impact of policies;
Definition: Includes performance of land use policies and potential
impact of transit project on regional land use.
Source: FTA.
[End of table]
[End of section]
Appendix VI: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Katherine Siggerud, (202) 512-2834, [Hyperlink, siggerudk@gao.gov];
Nikki Clowers, (202) 512-4010, [Hyperlink, clowersa@gao.gov].
Staff Acknowledgments:
In addition to the individuals named above, other key contributors to
this report were Bert Japikse, Jessica Lucas-Judy, Sara Ann Moessbauer,
and Gary Stofko.
[End of section]
Related GAO Products:
[End of section]
Highway and Transit Investments: Options for Improving Information on
Projects' Benefits and Costs and Increasing Accountability for Results.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-172]
Washington, D.C.: January 24, 2005.
Mass Transit: FTA Needs to Better Define and Assess Impact of Certain
Policies on New Starts Program.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-748]
Washington, D.C.: June 25, 2004.
Mass Transit: FTA Needs to Provide Clear Information and Additional
Guidance on the New Starts Ratings Process.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-701]
Washington, D.C.: June 23, 2003.
Mass Transit: Status of New Starts Program and Potential for Bus Rapid
Transit Projects.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02-840T]
Washington, D.C.: June 20, 2002.
Mass Transit: FTA's New Starts Commitments for Fiscal Year 2003.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02-603]
Washington, D.C.: April 30, 2002.
Mass Transit: FTA Could Relieve New Starts Program Funding Constraints.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-987]
Washington, D.C.: August 15, 2001.
Mass Transit: Implementation of FTA's New Starts Evaluation Process and
FY 2001 Funding Proposals.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/RCED-00-149]
Washington, D.C.: April 28, 2000.
Mass Transit: Challenges in Evaluating, Overseeing, and Funding Major
Transit Projects.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/T-RCED-00-104]
Washington, D.C.: March 8, 2000.
Mass Transit: "Mobility Improvements" Is One of Many Factors Used to
Evaluate Mass Transit Projects.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/RCED-00-6R]
Washington, D.C.: October 15, 1999.
Mass Transit: Status of New Starts Transit Projects With Full Funding
Grant Agreements.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/RCED-99-240]
Washington, D.C.: August 19, 1999.
Mass Transit: FTA's Progress in Developing and Implementing a New
Starts Evaluation Process.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/RCED-99-113]
Washington, D.C.: April 26, 1999.
(542050):
FOOTNOTES
[1] Fixed guideway systems use and occupy a separate right-of-way for
the exclusive use of public transportation services. They include fixed
rail, exclusive lanes for buses and other high-occupancy vehicles, and
other systems.
[2] This is the amount appropriated through fiscal year 2005, according
to FTA.
[3] Pub. L. No. 105-178, 112 Stat. 107 (1998).
[4] New Starts commitment authority is the amount of funding Congress
has authorized FTA to commit to New Starts projects for a given
authorization period.
[5] See the list of related GAO products at the end of this report.
[6] The fiscal year 2006 evaluation cycle began in May 2004, with the
issuance of the New Starts reporting instructions. Applications were
due in August 2004, and FTA's evaluation of the applications was
conducted in the fall of 2004. The annual report was published in
February 2005 and included funding recommendations for fiscal year
2006.
[7] The New Starts Working Group is an organization of New Starts
project sponsors, metropolitan planning organizations, and private
industry transit firms who advocate on behalf of the New Starts program
and specific projects.
[8] FTA rated 27 projects in the fiscal year 2006 cycle, but one of
these projects subsequently withdrew from the New Starts process;
therefore, we interviewed sponsors from 26 projects.
[9] "Guaranteed" funds are subject to a procedural mechanism designed
to ensure that a minimum amount of funding is made available each year
over the life of the project.
[10] During the preliminary engineering phase, project sponsors refine
the design of the proposal, taking into consideration all reasonable
design alternatives, which results in estimates of costs, benefits, and
impacts (e.g., financial or environmental). According to FTA officials,
to gain approval for entry into preliminary engineering, a project must
(1) have been identified through the alternatives analysis process, (2)
be included in the region's long-term transportation plan, (3) meet the
statutorily defined project justification and financial criteria, and
(4) demonstrate that the sponsors have the technical capability to
manage the project during preliminary engineering. Some federal New
Starts funding is available to projects for preliminary engineering
activities.
[11] Final design is the last phase of project development before
construction and may include right-of-way acquisition, utility
relocation, and the preparation of final construction plans and cost
estimates.
[12] The administration's funding recommendations are made in the
President's budget and are included in FTA's annual New Starts report
to Congress, which is released each February in conjunction with the
President's budget.
[13] FTA does not evaluate and rate projects with existing FFGAs, that
are in alternatives analysis, or that are statutorily exempt because
they are requesting less than $25 million in New Starts funding.
[14] FTA uses the incremental cost per hour of TSUB for assessing a
project's cost-effectiveness.
[15] For more information about the problems project sponsors
encountered in implementing these changes, see GAO, Mass Transit: FTA
Needs to Provide Clear Information and Additional Guidance on the New
Starts Ratings Process, GAO-03-701 (Washington, D.C.: June 23, 2003).
[16] See GAO, Mass Transit: FTA Needs to Better Define and Assess
Impact of Certain Policies on New Starts Program, GAO-04-748
(Washington, D.C.: June 25, 2004).
[17] While the total number of projects in preliminary engineering and
final design has fluctuated from year to year, this includes the
statutorily exempt projects. Once these projects are excluded from the
total, the number declines steadily each year.
[18] GAO-04-748.
[19] TEA-21 limits the amount of New Starts funding that can be used
for purposes other than final design and construction to not more than
8 percent of funds appropriated.
[20] The Surface Transportation Extension Act of 2005 (P.L. 109-14)
extended the programs until June 30, 2005.
[21] We compiled a list of changes that FTA has made to the
application, evaluation, rating, and project development oversight
processes from TEA-21, FTA regulations, annual reports and reporting
instructions, and previous GAO reports. We verified this list of
changes with FTA officials and project sponsors. We chose the fiscal
year 2000 evaluation cycle as our baseline because this was the first
cycle to reflect some TEA-21 changes. TEA-21 formalized many pre-
existing FTA practices, so we did not include those in our review. FTA
has also made other minor modifications to the New Starts process,
which we did not include in our review.
[22] This change was included in the formal rulemaking process
initiated after TEA-21 and was published as part of the New Starts
program regulations in December 2000.
[23] In response to project sponsor requests, in June 2005, FTA e-
mailed two examples of the "make the case" document to those project
sponsors who had registered for the 2005 New Starts roundtables.
[24] Both the TSUB and the Before and After study requirements were
included in the formal rulemaking process, and were implemented as part
of the New Starts program regulations in December 2000.
[25] For example, see GAO-04-748 and "The Rating and Evaluation of New
Starts Transit Systems," Statement of the Honorable Kenneth M. Mead,
Inspector General, U.S. Department of Transportation, before the
Committee on Appropriations, Subcommittee on Transportation, Treasury
and Independent Agencies, U.S. House of Representatives, April 28,
2004.
[26] The Department of Transportation publishes and stores on-line
information about proposed and final regulations, copies of public
comments on proposed rules, and related information on its Docket
Management System. The department uses this docketed material when
making regulatory and adjudicatory decisions, and makes docketed
material available for review by interested parties.
[27] 65 Fed. Reg. 76864 (Dec. 7, 2000). The Federal Transit Act of
1998, within TEA-21, required FTA to publish regulations on the manner
in which proposed projects will be evaluated and rated.
[28] At the time this report went to print, the House and Senate were
in conference and no conference report was available.
[29] H.R. 3, Sec. 3031, 109TH Cong. (2005).
[30] 151 Cong. Rec. S5668 (daily ed. May 20, 2005).
[31] FTA cited the delay in the implementation of the TSUB measure as
an example of FTA providing additional time for project sponsors to
comply with a change.
[32] See GAO, Highlights of a GAO Forum: Mergers and Transformation:
Lessons Learned for a Department of Homeland Security and Other Federal
Agencies, GAO-03-293SP (Washington, D.C.: Nov. 14, 2002).
[33] A webinar is a seminar or workshop that is conducted
electronically over the World Wide Web. A listserve is an electronic
mailing list that allows subscribers to send and receive information on
a particular topic.
[34] Specifically, when mobility improvements are rated "low," the
summary rating will "round down" to the lower of the two ratings; for
all other mobility improvement ratings, the rating is "rounded up" to
establish the summary project justification rating.
[35] 49U.S.C. 5309 (e)(6).
[36] U.S. Department of Transportation, Federal Transit Administration,
"Report to Congress on Evaluating New Starts Projects," January 3,
2005, p. 6.
[37] 65 Fed. Reg. 76875 (Dec. 7, 2000).
[38] 65 Fed. Reg. 76873 (Dec. 7, 2000).
[39] For the financial rating, the criteria also serve as the measures.
[40] The current mobility improvements and cost-effectiveness measure
(i.e., TSUB) does not include highway congestion relief benefits.
[41] The rating for the mobility improvements may be used as a tie-
breaker between cost-effectiveness and land use ratings--that is, a low
mobility improvements rating will round down the summary rating and a
mobility improvements rating above a low will cause the summary rating
to be rounded up. However, FTA officials told us that ties have been
rare in the last several years. The "other" factors will also be
considered if there is a compelling reason to do so; however, an FTA
official told us that to the best of his knowledge these other factors
have never changed a project's rating.
[42] For more information on double-counting the benefits of
transportation investments, see GAO, Highway and Transit Investments:
Options for Improving Information on Projects' Benefits and Costs and
Increasing Accountability for Results, GAO-05-172 (Washington, D.C.:
Jan. 24, 2005).
[43] The U.S. Environmental Protection Agency designates each region as
in attainment, nonattainment, or maintenance--reflecting current
compliance with the National Ambient Air Quality Standards under the
Clean Air Act--for transportation-related pollutants including ozone,
carbon monoxide, particulate matter, and nitrogen oxides. Geographic
areas that have levels of a pollutant above those allowed by the
standard are called nonattainment areas. Areas that did not meet the
standard for a pollutant in the past but have reached attainment and
met certain procedural requirements are known as maintenance areas.
[44] "The Rating and Evaluation of New Starts Transit Systems,"
Statement of the Honorable Kenneth M. Mead, Inspector General, U.S.
Department of Transportation, before the Committee on Appropriations,
Subcommittee on Transportation, Treasury and Independent Agencies, U.S.
House of Representatives, April 28, 2004.
[45] GAO-05-172.
[46] FTA introduced the Summit software in the fiscal year 2004 rating
cycle to improve the accuracy of local travel models used to support
New Starts projects. According to FTA, Summit has provided a means to
identify and diagnose travel forecasting problems related to
assumptions regarding fare and service policies, regional
transportation networks, land use, and economic conditions. The
software has also helped ensure that local forecasts are utilizing
comprehensive and up-to-date data on travel behavior and local
transportation systems.
[47] See GAO, Tax Administration: IRS Needs to Further Refine Its Tax
Filing Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov.
22, 2002).
[48] GAO, Mass Transit: FTA Needs to Better Define and Assess Impact of
Certain Policies on New Starts Program, GAO-04-748 (Washington, D.C.:
June 25, 2004).
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
