Federal Research
Observations on the Small Business Innovation Research Program Gao ID: GAO-05-861T June 28, 2005Since it was established in 1982, GAO has consistently reported on the success of the Small Business Innovation Research (SBIR) program in benefiting small, innovative companies, strengthening their role in federal research and development (R&D), and helping federal agencies achieve their R&D goals. However, through these reviews GAO has also identified areas where action by participating agencies or the Congress could build on the program's successes and improve its operations. This statement for the record summarizes the program's successes and improvements over time, as well as the continuing challenge of assessing the long term results of the program.
Between July 1985 and June 1999, GAO reviewed, reported, and testified on the SBIR program many times at the request of the Congress. While GAO's work focused on many different aspects of the program, it generally found that SBIR is achieving its goals to enhance the role of small businesses in federal R&D, stimulate commercialization of research results, and support the participation of small businesses owned by women and/or disadvantaged persons. Participating agencies and companies that GAO surveyed during the course of its reviews generally rated the program highly. GAO also identified areas of weaknesses and made recommendations that, if addressed, could strengthen the program further. Some of these concerns related to (1) duplicate funding for similar, or even identical, research projects by more than one agency, (2) inconsistent interpretations of extramural research budgets by participating agencies, (3) geographical concentration of awards in a small number of states, and (4) lack of clarification on the emphasis that agencies should give to a company's commercialization record when assessing its proposals. Most of GAO's recommendations for program improvement have been either fully or partially addressed by the Congress in various reauthorizations of the program or by the agencies themselves. One issue that continues to remain somewhat unresolved after almost two decades of program implementation is how to assess the performance of the SBIR program. As the program has matured, the Congress has emphasized the potential for commercialization as an important criterion in awarding funds and the commercialization of a product as a measure of success for the program. However, in 1999, GAO reported that the program's other goals also remain important to the agencies. By itself, according to some program managers, limited commercialization may not signal "failure" because a company may have achieved other goals, such as innovation or responsiveness to an agency's research needs. GAO identified a variety of reasons why assessing the performance of the SBIR program has remained a challenge. First, because the authorizing legislation and the Small Business Administration's (SBA) policy directives do not define the role of the company's commercialization record in determining commercial potential and the relative importance of the program's goals, different approaches have emerged in agencies' evaluations of proposals. Second, GAO found that it has been difficult to find practical ways to define and measure the SBIR program's goals in order to evaluate proposals. For example, the authorizing legislation lacks a clear definition of "commercialization," and agencies sometimes differed on its meaning. Finally, GAO reported that as the emphasis on commercialization had grown, so had concerns that noncommercial successes may not be adequately recognized. For example, program managers identified various projects that met special military or medical equipment needs but that had limited sales potential.
GAO-05-861T, Federal Research: Observations on the Small Business Innovation Research Program
This is the accessible text file for GAO report number GAO-05-861T
entitled 'Federal Research: Observations on the Small Business
Innovation Research Program' which was released on June 29, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Environment, Technology, and Standards,
Committee on Science, House of Representatives:
United States Government Accountability Office:
GAO:
For Release on Delivery Expected at 3:00 p.m. EDT:
Tuesday, June 28, 2005:
Federal Research:
Observations on the Small Business Innovation Research Program:
Statement for the Record of Anu K. Mittal, Director:
Natural Resources and Environment Team:
GAO-05-861T:
GAO Highlights:
Highlights of GAO-05-861T, a report to Subcommittee on Environment,
Technology and Standards, Committee on Science, House of
Representatives:
Why GAO Did This Study:
Since it was established in 1982, GAO has consistently reported on the
success of the Small Business Innovation Research (SBIR) program in
benefiting small, innovative companies, strengthening their role in
federal research and development (R&D), and helping federal agencies
achieve their R&D goals. However, through these reviews GAO has also
identified areas where action by participating agencies or the Congress
could build on the program‘s successes and improve its operations. This
statement for the record summarizes the program‘s successes and
improvements over time, as well as the continuing challenge of
assessing the long term results of the program.
What GAO Found:
Between July 1985 and June 1999, GAO reviewed, reported, and testified
on the SBIR program many times at the request of the Congress. While
GAO‘s work focused on many different aspects of the program, it
generally found that SBIR is achieving its goals to enhance the role of
small businesses in federal R&D, stimulate commercialization of
research results, and support the participation of small businesses
owned by women and/or disadvantaged persons. Participating agencies and
companies that GAO surveyed during the course of its reviews generally
rated the program highly.
GAO also identified areas of weaknesses and made recommendations that,
if addressed, could strengthen the program further. Some of these
concerns related to (1) duplicate funding for similar, or even
identical, research projects by more than one agency, (2) inconsistent
interpretations of extramural research budgets by participating
agencies, (3) geographical concentration of awards in a small number of
states, and (4) lack of clarification on the emphasis that agencies
should give to a company‘s commercialization record when assessing its
proposals. Most of GAO‘s recommendations for program improvement have
been either fully or partially addressed by the Congress in various
reauthorizations of the program or by the agencies themselves.
One issue that continues to remain somewhat unresolved after almost two
decades of program implementation is how to assess the performance of
the SBIR program. As the program has matured, the Congress has
emphasized the potential for commercialization as an important
criterion in awarding funds and the commercialization of a product as a
measure of success for the program. However, in 1999, GAO reported that
the program‘s other goals also remain important to the agencies. By
itself, according to some program managers, limited commercialization
may not signal ’failure“ because a company may have achieved other
goals, such as innovation or responsiveness to an agency‘s research
needs. GAO identified a variety of reasons why assessing the
performance of the SBIR program has remained a challenge. First,
because the authorizing legislation and the Small Business
Administration‘s (SBA) policy directives do not define the role of the
company‘s commercialization record in determining commercial potential
and the relative importance of the program‘s goals, different
approaches have emerged in agencies‘ evaluations of proposals. Second,
GAO found that it has been difficult to find practical ways to define
and measure the SBIR program‘s goals in order to evaluate proposals.
For example, the authorizing legislation lacks a clear definition of
’commercialization,“ and agencies sometimes differed on its meaning.
Finally, GAO reported that as the emphasis on commercialization had
grown, so had concerns that noncommercial successes may not be
adequately recognized. For example, program managers identified various
projects that met special military or medical equipment needs but that
had limited sales potential.
www.gao.gov/cgi-bin/getrpt?GAO-05-861T.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Anu K. Mittal, (202) 512-
3841, mittala@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
We are pleased to have the opportunity to comment on the Small Business
Innovation Research (SBIR) program. Since the program's inception, we
have consistently reported on its success in benefiting small,
innovative companies, strengthening their role in federal research and
development (R&D), and helping federal agencies achieve their R&D
goals. However, through these reviews we have also identified areas
where action by participating agencies or the Congress could build on
the program's successes and improve its operations. Over the life of
the program these recommendations have largely been implemented. This
statement will discuss the program's successes as well as the
continuing challenge of assessing the long term results of the SBIR
program.
As a competitor in the global economy, the United States relies heavily
on innovation through research and development. The potential of small
businesses to be sources of significant innovation led the Congress to
increase government funding for R&D projects with commercial potential
that are conducted by small high-technology companies. In this context,
the Small Business Innovation Development Act of 1982 established the
SBIR program to stimulate mission-related technological innovation, use
small businesses to meet federal R&D needs, foster participation by
minority and disadvantaged persons in technological innovation, and
increase private sector commercialization of innovations derived from
federal R&D[Footnote 1]. The act provided for a three-phased program:
phase I to determine the feasibility and scientific and technical merit
of a proposed research idea; phase II to further develop the idea,
taking into account its commercial potential; and phase III to
commercialize the resulting product or process with no further SBIR
funding.
The original program was reauthorized in 1986, extending the program's
expiration date from 1988 to 1993.[Footnote 2] In 1992, it was
reauthorized by the Small Business Research and Development Enhancement
Act to expand and improve the program, to emphasize its goal of
increasing private sector commercialization, to increase participation
by small businesses, and to improve the government's dissemination of
program-related information.[Footnote 3] In addition, the act increased
funding for phase-I and phase-II proposals to $100,000 and $750,000,
respectively, with adjustments once every 5 years for inflation and
changes in the program. The program was again reauthorized in 2000 by
the Consolidated Appropriations Act of 2001,[Footnote 4] which directed
the Small Business Administration (SBA) and participating agencies to,
among other things, expand the scope of publicly available information
on specific grants and to annually report on their SBIR programs. In
addition, the act requires award recipients to provide information to
help SBA evaluate the program. The SBIR program is currently scheduled
to expire on September 30, 2008.
Current law requires every federal department with an R&D budget of
$100 million or more to establish and operate a SBIR program funded by
a set percentage of that agency's extramural R&D budget--originally
1.25 percent and now 2.5 percent. In addition, agencies with R&D
spending above $20 million are directed to establish goals for
financing small business R&D projects at levels higher than the
previous year. As of fiscal year 2004, 12 federal agencies participated
in the SBIR program, including the departments of Agriculture,
Commerce, Defense (DOD), Education, Energy, Health and Human Services
(HHS), Homeland Security, Housing and Urban Development, and
Transportation; the Environmental Protection Agency; the National
Aeronautics and Space Administration (NASA); and the National Science
Foundation (NSF). Each agency manages its own program, while SBA plays
a central administrative role, such as issuing policy directives and
annual reports for the program. Awards from three agencies--DOD,
National Institutes of Health, and NASA--account for the majority of
SBIR funds. From its inception in fiscal year 1983 through fiscal year
2003, federal agencies have awarded over $15 billion for more than
76,000 projects.
SBIR Program Has Generally Met Its Goals:
Between July 1985 and June 1999, we reviewed, reported, and testified
on the SBIR program many times at the request of the Congress. While
our work focused on many different aspects of the program, we generally
found that SBIR is achieving its goals to enhance the role of small
businesses in federal R&D, stimulate commercialization of research
results, and support the participation of small businesses owned by
women and/or disadvantaged persons. Participating agencies and
companies that we surveyed during the course of our reviews generally
rated the program highly. Specific examples of program success that we
identified include the following:
* High-quality research. Throughout the life of the program, awards
have been based on technical merit and are generally of good quality.
For example, in 1989 we reported that according to agency officials,
more than three-quarters of the research conducted with SBIR funding
was as good as or better than other agency-funded research. Agency
officials also rated the research as more likely than other research
they oversaw to result in the invention and commercialization of new
products. When we again looked at the quality of research proposals in
1995, we found that while it was too early to make a conclusive
judgment about the long-term quality of the research, the quality of
proposals remained good, according to agency officials.
* Widespread competition. The SBIR program successfully attracts many
qualified companies, has had a high level of competition, and
consistently has had a high number of first-time participants.
Specifically, we reported that the number of proposals that agencies
received each year had been increasing. In addition, as we reported in
1998, agencies rarely received only a single proposal in response to a
solicitation, indicating a sustained level of competition for the
awards. We also found that the agencies deemed many more proposals
worthy of awards than they were able to fund. For example, the Air
Force deemed 1,174 proposals worthy of awards in fiscal year 1993 but
funded only 470. Moreover, from fiscal years 1993 through 1997, one
third of the companies that received awards were first-time
participants. This suggests that the program attracts hundreds of new
companies annually.
* Effective outreach. SBIR agencies consistently reach out to foster
participation by women-owned or socially and economically disadvantaged
small businesses. For example, we found that DOD's SBIR managers
participated in a number of regional small business conferences and
workshops that are specifically designed to foster increased
participation by women-owned and socially and economically
disadvantaged small businesses.
* Successful commercialization. SBIR successfully fosters
commercialization of research results. At various points in the life of
the program we have reported that SBIR has been successful in
increasing private sector commercialization of innovations. For
example, past GAO and DOD surveys of companies that received SBIR Phase
II funding have determined that approximately 35 percent of the
projects resulted in the sales of products or services, and
approximately 45 percent of the projects received additional
developmental funding. We have also reported that agencies were using
various techniques to foster commercialization. For example, in an
attempt to get those companies with the greatest potential for
commercial success to the marketplace sooner, DOD instituted a Fast
Track Program, whereby companies that are able to attract outside
commitments/capital for their research during phase I are given higher
priority in receiving a phase II award.
* Helping to serve mission needs. SBIR has helped serve agencies'
missions and R&D needs. Agencies differ in the emphasis they place on
funding research to support their mission and to support more
generalized research. Specifically, we found that DOD links its
projects more closely to its mission. In comparison, other agencies
emphasize research that will be commercialized by the private sector.
Many of the projects DOD funded have specialized military applications
while NIH projects have access to the biomedical market in the private
sector. Moreover, we found that SBIR promotes research on the critical
technologies identified in lists developed by DOD and/or the National
Critical Technologies Panel. Generally agencies reviewed these listings
of critical technologies to develop research topics or conducted
research that fell within one of the two lists.
Improvements Made to the SBIR Program Over Time:
We have also identified areas of weaknesses and made recommendations
that, if addressed, could strengthen the program further. Many of our
recommendations for program improvement have been either fully or
partially addressed by the Congress in various reauthorizations of the
program or by the agencies themselves. For example,
* Duplicate funding. In 1995, we identified duplicate funding for
similar, or even identical, research projects by more than one agency.
A few companies received funding for the same proposals two, three, and
even five times before agencies became aware of the duplication.
Contributing factors included the fraudulent evasion of disclosure by
companies applying for awards, the lack of a consistent definition for
key terms such as "similar research," and the lack of interagency
sharing of data on awards. In response to our recommendations, SBA
strengthened the language agencies use in their application packages to
clearly warn applicants about the illegality of entering into multiple
agreements for essentially the same effort and developed Internet
capabilities to access SBIR data for all of the agencies. In SBA's
view, the stronger language regarding the illegality of seeking funding
for similar or identical projects addresses the need to develop
consistent definitions to help agencies determine when projects are
"similar."
* Inconsistent interpretations of extramural research budgets. In 1998,
we found that while agency officials adhered to SBIR's program and
statutory funding requirements, they used differing interpretations of
how to calculate their "extramural research budgets." As a result some
agencies were inappropriately including or excluding some types of
expenses. To address our recommendation that SBA provide additional
guidance on how participating agencies were to calculate their
extramural research budgets, the Congress in 2000 required that the
agencies report annually to SBA on the methods used to calculate their
extramural research budgets.
* Geographical concentration of awards. In 1999, in response to
congressional concerns about the geographical concentration of SBIR
awards, we reported that companies in a small number of states,
especially California and Massachusetts, have submitted the most
proposals and won the majority of awards. The distribution of awards
generally followed the pattern of distribution of non-SBIR expenditures
for R&D, venture capital investments, and academic research funds. We
reported that some agencies had undertaken efforts to broaden the
geographic distribution of awards and that the program implemented by
the National Science Foundation had been particularly effective.
Although we did not make any recommendations on how to improve the
program's outreach to states receiving fewer awards, in the 2000
reauthorization of the program, Congress established the Federal and
State Technology Partnership Program to help strengthen the
technological competitiveness of small businesses, especially in those
states that receive fewer SBIR grants.
* Clarification on commercialization and other SBIR goals. Finally, in
response to our continuing concern that clarification was needed on the
relative emphasis that agencies should give to a company's
commercialization record and SBIR's other goals when evaluating
proposals, in 2000 the Congress required companies applying for a
second phase award to include a commercialization plan with their SBIR
proposals. This requirement partially addressed our concern. Moreover,
in the spring of 2001, SBA initiated efforts to respond to our
recommendation to develop standard criteria for measuring commercial
and other outcomes of the SBIR program, such as uniform measures of
sales and developmental funding, and incorporate these criteria into
its Tech-Net database. Specifically, SBA began implementing a reporting
system to measure the program's commercialization success. In fiscal
year 2002, SBA further enhanced the reporting system to include
commercialization results that would help establish an initial baseline
rate of commercialization. In addition, small business firms
participating in the SBIR program are required to provide information
annually on sales and investments associated with their SBIR projects.
Assessing the Performance of the SBIR Program Remains a Challenge:
One issue that continues to remain somewhat unresolved after almost two
decades of program implementation is how to assess the performance of
the SBIR program. As the program has matured, the Congress has
emphasized the potential for commercialization as an important
criterion in awarding funds and the commercialization of a product as a
measure of success for the program. However, in 1999, we reported that
the program's other goals also remain important to the agencies. By
itself, according to some program managers, limited commercialization
may not signal "failure" because a company may have achieved other
goals, such as innovation or responsiveness to an agency's research
needs. We identified a variety of reasons why assessing the performance
of the SBIR program has remained a challenge.
* First, because the authorizing legislation and SBA's policy
directives do not define the role of the company's commercialization
record in determining commercial potential and the relative importance
of the program's goals, different approaches have emerged in agencies'
evaluations of proposals. As a result, the relative weight that should
be given to the program's goals when evaluating proposals remains
unclear. Innovation and responsiveness to an agency's needs, for
example, may compete with the achievement of commercialization. In the
view of many program managers, innovation involves a willingness to
undertake R&D with a higher element of risk and a greater chance that
it may not lead to a commercial product; responsiveness to an agency's
needs involves R&D that may be aimed at special niches with limited
commercial potential. Striking the right balance between achieving
commercial sales and encouraging new, unproven technologies is,
according to the program managers, one of the key ingredients in the
program's overall success.
* Second, we found that it has been difficult to find practical ways to
define and measure the SBIR program's goals in order to evaluate
proposals. For example, the authorizing legislation lacks a clear
definition of "commercialization," and agencies sometimes differed on
its meaning. This absence of a definition makes it more difficult to
determine when a frequent winner is "failing" to achieve a sufficient
level of commercialization and how to include this information in an
agency's review of the company's proposal. Similarly, efforts to define
and measure technological innovation, which was one of the program's
original goals, have posed a challenge. Although definitions vary,
there is widespread agreement that technological innovation is a
complex process, particularly in the development of sophisticated
modern technologies.
* Finally, we reported that as the emphasis on commercialization had
grown, so had concerns that noncommercial successes may not be
adequately recognized. For example, program managers identified various
projects that met special military or medical equipment needs but that
had limited sales potential. These projects would be helpful in
reducing the agency's expenditures and meeting the mission of the
agency but may not be appropriately captured in typical measurements of
commercialization. In general, we found that program managers valued
both noncommercial and commercial successes and feared that the former
might be ignored in emphasizing the latter.
To help evaluate the performance of the program, in the 2000
reauthorization of SBIR, Congress required SBA to develop a database
that would help the agency collect and maintain in common format
necessary program output and outcome information. The database is to
include the following information on all phase II awards: (1) revenue
from the sale of new products or services resulting from the SBIR
funded research, (2) additional investment from any non-SBIR source for
further research and development, and (3) any other description of
outputs and outcomes of the awards. In addition, the database is to
include general information for all applicants not receiving an award
including an abstract of the project.
In conclusion, Mr. Chairman, our work has shown that, overall, the SBIR
program has been successful in meeting its goals and that the Congress
and the agencies have implemented actions to strengthen the program
over time. However, an assessment of the program's results remains a
challenge because of the lack of clarity on how much emphasis the
program should place on commercialization versus other goals.
For further information, please contact Anu Mittal at (202) 512-3841 or
mittala@gao.gov.
FOOTNOTES
[1] Pub.L. No. 97-219 (1982).
[2] Pub.L. No. 99-443 (1986).
[3] Pub.L. No. 102-564 (1992).
[4] Pub. L. No. 106-554 (2000).
