Electronic Procurement
Business Strategy Needed for GSA's Advantage System Gao ID: GAO-03-328 February 19, 2003The Congress has been seeking to increase federal agency purchases of equipment and services on line. The General Services Administration's (GSA) Advantage Internet-based ordering system is meant for conducting market research and ordering all types of products and services on line. About $84 million has already been invested in this endeavor. GAO was asked to assess how effectively GSA has managed this investment.
GSA Advantage has had only limited success as an on-line market research and ordering tool. Market research has been limited primarily to off-the-shelf office products, and sales through Advantage have never exceeded one-half of 1 percent of overall schedule sales. Because of initial design limitations, Advantage has not been effective in acquiring complex products and services, particularly information technology services that make up most of the growth in schedule sales. Recent enhancements may facilitate use of the system for buying complex services, but vendor participation has not been strong. For example, for almost half the requests for quotes, agencies received no responses from vendors. Advantage does not contain all schedule products and services, as required, and some schedule vendors complain about the cost and complexity involved in participating. While GSA has modified the Advantage system to provide better and more information on products and services, the system is unlikely to succeed without a business strategy. To date, GSA has not yet answered basic questions surrounding Advantage, including (1) if it is technically feasible to make purchases of all products and services on line given the range of activities involved with making more complex acquisitions, (2) if so, whether Advantage is the right system to do this, and (3) if not, whether Advantage could be used for smaller, simpler purchases and as an information source for making larger, more complicated acquisitions. Moreover, GSA has not thoroughly analyzed other alternatives to Advantage or developed effective measures to assess return on investment and user satisfaction. A business strategy would require GSA to not only reexamine its objectives for on-line purchasing and various solutions, but also develop measures appropriate for gauging success. As such, it would help GSA assess whether continued investment in Advantage is worthwhile and make modifications to the system that will substantially enhance its performance.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-328, Electronic Procurement: Business Strategy Needed for GSA's Advantage System
This is the accessible text file for GAO report number GAO-03-328
entitled 'Electronic Procurement: Business Strategy Needed for GSA's
Advantage System' which was released on March 21, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products‘ accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
GAO:
Report to the Chairman, Committee on Government Reform, U.S. House of
Representatives:
February 2003:
Electronic Procurement:
Business Strategy Needed for GSA‘s Advantage System:
GAO-03-328:
GAO Highlights:
Highlights of GAO-03-328, a report to the Chairman, Committee on
Government Reform, U.S. House of Representatives:
February 2003:
Electronic Procurement:
Business Strategy Needed for GSA‘s Advantage System:
Why GAO Did This Study:
The Congress has been seeking to increase federal agency purchases of
equipment and services on line. The General Services Administration‘s
(GSA) Advantage Internet-based ordering system is meant for conducting
market research and ordering all types of products and services on
line.
About $84 million has already been invested in this endeavor. GAO was
asked to assess how effectively GSA has managed this investment.
What GAO Found:
GSA Advantage has had only limited success as an on-line market
research
and ordering tool. Market research has been limited primarily to off-
the-
shelf office products, and sales through Advantage have never exceeded
one–
half of 1 percent of overall schedule sales. Because of initial design
limitations, Advantage has not been effective in acquiring complex
products
and services, particularly information technology services that make up
most of the growth in schedule sales. Recent enhancements may
facilitate
use of the system for buying complex services, but vendor participation
has not been strong. For example, for almost half the requests for
quotes,
agencies received no responses from vendors. Advantage does not
contain
all schedule products and services, as required, and some schedule
vendors
complain about the cost and complexity involved in participating.
While GSA has modified the Advantage system to provide better and more
information on products and services, the system is unlikely to succeed
without a business strategy. To date, GSA has not yet answered basic
questions surrounding Advantage, including (1) if it is technically
feasible
to make purchases of all products and services on line given the range
of
activities involved with making more complex acquisitions, (2) if so,
whether Advantage is the right system to do this, and (3) if not,
whether
Advantage could be used for smaller, simpler purchases and as an
information
source for making larger, more complicated acquisitions.
Moreover, GSA has not thoroughly analyzed other alternatives to
Advantage
or developed effective measures to assess return on investment and user
satisfaction. A business strategy would require GSA to not only
reexamine
its objectives for on-line purchasing and various solutions, but also
develop
measures appropriate for gauging success. As such, it would help GSA
assess
whether continued investment in Advantage is worthwhile and make
modifications
to the system that will substantially enhance its performance.
Figure:
[See PDF for image]
[End of figure]
What GAO Recommends:
GAO recommends that GSA develop a business strategy that reexamines its
objectives for on-line purchasing of products and services, assesses
what
solutions are most appropriate for meeting these objectives, and
develops
measures for assessing success. GSA concurs with this report‘s
recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-03-328.
To view the full report, including the scope and methodology, click on
the
link above.For more information, contact William Woods (202) 512-4841,
WoodsW@gao.gov
contents:
Letter:
Results in Brief:
Background:
Agencies Use Advantage More for Market Research than for Ordering, and
Mostly
for Standard Office Products:
Accomodating Buys of Complex Services Has Been Problematic:
Lack of Full Vendor Participation Due to Concerns About Cost and Use:
Performance Reflects the Lack of a Business Strategy:
Conclusion:
Recommendations:
Agency Comments:
Scope and Methodology:
Appendix:
Comments from the General Services Administration:
Figure:
Figure 1: Sales under FSS Schedule Contracts--Fiscal Years 1998 to
2002:
This is a work of the U.S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its
entirety without further permission from GAO. It may contain
copyrighted
graphics, images or other materials. Permission from the copyright
holder
may be necessary should you wish to reproduce copyrighted material
separately from GAO‘s product.
February 19, 2003:
The Honorable Tom Davis:
Chairman:
Committee on Government Reform:
House of Representatives:
Dear Mr. Chairman:
Federal agencies ordered about $22 billion worth of products and
services from commercial firms during fiscal year 2002 using the
General Services Administration‘s (GSA) Multiple Award Schedules
program. To help federal agencies use the program more efficiently, GSA
developed a system called GSA Advantage, which is an Internet-based
system for conducting market research and ordering schedule products
and services on line. Because of concerns about the effectiveness of
GSA Advantage, you asked that we review the performance of the system.
In response to your request, we assessed how effective Advantage has
been as an on-line system for federal agencies to shop for and order
schedule products and services.
Results in Brief:
GSA Advantage has had only limited success as an on-line market
research and ordering tool. Market research has been limited primarily
to off-the-shelf office products, and sales through Advantage have
never exceeded one-half of 1 percent of overall schedule sales. Because
of initial design limitations, Advantage has not been an effective
means for acquiring complex products and services, particularly
information technology services that make up most of the growth in
schedule sales. Recent enhancements may facilitate use of the system
for buying complex services, but vendor participation has not been
strong. For example, for almost half the requests for quotes, agencies
received no responses from vendors. Advantage does not contain all
schedule products and services, as required, and some schedule vendors
complain about the cost and complexity involved in participating.
The limited success of Advantage reflects the lack of a business
strategy for the system. Such a strategy normally would examine such
issues as what type of schedule products and services is best suited
for on-line market research and ordering, whether Advantage is the best
system to accomplish these functions in a manner consistent with the
needs of customer agencies and the capabilities of the vendors, and how
the performance of that system could be measured. After developing such
a strategy, GSA would have greater assurance that its system is the
most feasible solution to accomplish its objectives and would have a
sound basis for setting policies for Advantage. We are recommending
that GSA develop a comprehensive business strategy for the Advantage
system.
Background:
GSA‘s mission includes providing policy leadership and expertise in the
acquisition of products and services for use by federal agencies. To
help carry out its mission, GSA‘s Federal Supply Service (FSS) assists
federal agencies in acquiring supplies --such as furniture, computers,
tools, and equipment --and a variety of services through its commercial
acquisition business line. This GSA business line gives agencies access
to several million commercial products and services available either
through schedule contracts or as stock items from a GSA warehouse.
GSA launched Advantage in October 1995. The system was envisioned as an
on-line system for agencies to order stock items from GSA‘s warehouses
with a government payment card. Initially, Advantage offered only stock
items. In February 1996, Congress passed the Clinger-Cohen Act, which
required the Administrator of GSA to develop a system capable of
providing governmentwide, on-line Internet access to information on all
FSS schedule products and services. GSA decided that Advantage,
although in its infancy, was the only Internet-based, on-line ordering
system that potentially could meet the requirements of the
act,[Footnote 1] and began using Advantage to satisfy this requirement.
GSA began adding schedule services to Advantage in February 2000.
Despite efforts to include more products and services on Advantage,
only 35 percent of all vendors participated in the system at the
beginning of fiscal year 2001. GSA therefore issued a mandate that all
schedule vendors participate in Advantage and offer all their products
and services through it. For off-the-shelf products, vendors typically
upload three types of information: product price, features, and
performance. In some cases, they offer quantity discounts. For
services, vendors typically upload the labor rates contained in their
schedule contracts, along with other information.
GSA has added various functions and enhancements to Advantage during
the last 7 years, such as reference files to increase the amount of
information available to customer agencies. In total, GSA has spent
about $84 million to develop, implement, and maintain Advantage.
Agencies Use Advantage More for Market Research than for Ordering, and
Mostly for Standard Office Products:
Customer agencies do a great deal more comparative shopping than
ordering through Advantage. During a 7-month period in fiscal year
2002, comparative shopping on Advantage ranged between 27,000 and
50,000 site searches each day.[Footnote 2] Most of this market research
involved office products and equipment. For example, the top searched-
for products included office supplies like binders, pens, copier paper,
and envelopes, and office equipment such as laptops, fax machines,
chairs, and shredders. Searches for services were far less frequent
than for products, and most often were for training, security,
maintenance, and consulting.
Sales through Advantage represent a very small portion of total
schedule sales. As shown in figure 1, overall schedule sales have grown
considerably in the past 5 years--from about $8.5 billion to almost $22
billion. Sales, however, have never exceeded one-half of 1 percent of
total schedule sales. For fiscal year 2002, for example, there were
129,370 schedule orders through Advantage for a total of $81,579,970,
representing 0.37 percent of total schedule sales. Most sales through
Advantage were for standard office products. For example, for fiscal
year 2002, the top 10 schedule items in order of sales frequency
included file folders, self-stick pads, multiuse paper, double pocket
portfolio, ink jet toner, copy paper, binder clips, and three-ring
binders. Our review of Advantage orders from 1997 through 2002 shows a
similar pattern of purchases of standard office products. According to
GSA, schedule sales through Advantage have averaged about $500 per
order.[Footnote 3]
Figure 1: Sales under FSS Schedule Contracts--Fiscal years 1998-2002:
[See PDF for image]
Source: GSA:
Note: Numbers on top of columns are the percentages of schedule sales
through Advantage:
[End of figure]
Accommodating Buys of Complex Services Has Been Problematic:
As shown in figure 1, most of the growth in overall schedule sales has
been in spending for information technology (IT) under GSA‘s schedule
70 contracts. Most of this growth in IT spending has been for IT
services, which now account for about 65 percent of IT spending. Yet an
inherent limitation in using Advantage is that the system was not
designed for making complex purchases, such as procurements of IT
services.
The information available on Advantage regarding complex services
typically consists of general descriptions of labor categories, such as
a program manager or network analyst, and loaded labor rates.[Footnote
4] In a prior report, we pointed out that this information alone is not
sufficient to enable agencies using GSA‘s schedules to select the best
value provider of IT services.[Footnote 5] Rather, agencies must
consider the number of hours and mix of labor categories proposed by
the schedule contractors to perform specific tasks. Special ordering
procedures issued by GSA require agencies acquiring complex services to
obtain this information by providing a statement of work to at least
three schedule contractors asking for either fixed or ceiling prices
for the work specified.
In 2001, GSA added a capability to Advantage, known as E-buy, which
enables agencies to provide a statement of work to vendors and request
responses on line. This modification provides a means to share
information between customer agencies and vendors on a range of
products and services, including the more complex IT services. During
fiscal year 2002, the first full fiscal year of operation, agencies
used E-buy to place a total of 64 orders. By comparison, customers
placed an average of about 1,240 non-E-buy orders each day through
Advantage during the same period.
Due to the limited initial use of E-buy, GSA revised the function in
July 2002 to make it more user-friendly. During the next 2 months of
operation (July-August 2002), agency usage of the revised system
increased, but there were other implementation problems. About 12
percent of requests for quotations (RFQ) were sent to fewer than three
vendors, as required. For each RFQ issued, agencies received an average
of only 1.3 responses. For almost half the RFQs (about 46 percent), the
agencies received no responsive quote from any vendor.
GSA officials have expressed disappointment in the low usage of E-buy
in its first year of operation, and they are concerned about the lack
of vendor responses following revisions to the system. The officials
also pointed out, however, that Advantage and its E-buy enhancement
should improve agency buyers‘ market research capabilities.
Full Vendor Participation Lacking Due to Concerns About Cost and Use:
Despite GSA‘s mandatory vendor participation policy, not all vendors
list all their products and services on Advantage. Out of approximately
9,800 schedule contracts, about 1,200 contracts (12 percent) were not
on Advantage as of September 2002. Even some vendors who supply off-
the-shelf products - which are most likely to be ordered on line -
decline to participate. For example, one vendor who sells scales and
butcher supplies is not on Advantage because the vendor‘s primary
customers do not use the system. Another vendor indicated that it
uploaded only certain products from its catalog; it has not uploaded
the services it offers because its principal service customers do not
use Advantage. Other vendors report that it can cost several thousand
dollars to upload data on Advantage, particularly for complex products
and services. Such expenditures do not make sense, they say, if
customers are not using Advantage to make purchases.
An underlying reason why some vendors are reluctant to list all of
their complex products and services on Advantage is because the system
does not contain sufficient descriptive information to permit agencies
to make informed buying decisions. To address this issue, GSA added a
capability called e-catalog reference files, which allows vendors to
supplement the information about their products or services by
providing links to their own Web pages. Although this capability
provides more information about vendor products and services, the
descriptions vary widely in quality.
Performance Reflects the Lack of a Business Strategy:
GSA has not conducted the type of analysis central to developing a
comprehensive business strategy for Advantage. Specifically, GSA has
not analyzed customer requirements, identified the products and
services most likely to be purchased on-line by agency customers,
assessed the technological feasibility of meeting customer requirements
through an on-line system, or considered alternatives to Advantage. In
addition, GSA has not established performance goals for Advantage or a
system for measuring results. Each of these is an important element of
a comprehensive business strategy.
GSA officials acknowledge the lack of a business strategy for
Advantage. According to these officials, GSA started Advantage before
performing the kind of rigorous analysis it would undertake before
initiating a major system today. They recognized that the benefit of
doing a business strategy was that it would have enabled GSA management
to answer two important questions: (1) how much and what kind of
products and services could be expected to be purchased by federal
customers on-line, and (2) whether Advantage is the right system to
facilitate such on-line purchases.
Conclusion:
GSA has spent about $84 million on Advantage, but it has not developed
a comprehensive business strategy that would justify that level of
investment. In the absence of a business strategy, GSA has made changes
to enhance the system, but is yet to target what products and services
are best suited for on-line market research and ordering. GSA has made
these modifications without integrating them into an overall strategy.
A rigorously developed business strategy, including performance
measures, would help GSA assess whether its continued investment in
Advantage is worthwhile and, if so, what additional changes to enhance
its performance may be needed.
Recommendation:
We recommend that the Administrator of General Services develop a
comprehensive business strategy that identifies GSA‘s objectives for an
on-line market research and ordering system, the desired capabilities
of that system, the resources required to implement the system, and the
measures to track performance. After developing the business strategy,
GSA should determine whether Advantage is the most appropriate system
for achieving its objectives.
Agency Comments:
GSA concurs with our recommendation. GSA‘s comments appear in Appendix
I.
Scope and Methodology:
To address our objective, we interviewed GSA and industry officials,
analyzed electronic files of past market research and on-line sales,
attended a GSA training course on how vendors electronically transfer
information about their products and services into Advantage, and
received various demonstrations of Advantage operations and marketing
techniques. We also visited GSA‘s two largest acquisition centers
(Arlington, Virginia and Fort Worth, Texas) to learn more about GSA‘s
efforts to address vendor concerns and encourage vendor participation
in Advantage. We conducted our review in accordance with generally
accepted government auditing standards from March through December
2002.
Unless you publicly announce the contents of this report earlier, we
plan no further distribution of it until 30 days from the report date.
At that time, we will send copies to other interested congressional
committees; the Administrator of General Services; and the Director,
Office of Management and Budget. We will also make copies available to
others upon request. In addition, the report will be available at no
charge on the GAO Web site at http://www.gao.gov.
Please contact me at (202) 512-8214 or Ralph Dawn at (202) 512-4544 if
you have any questions regarding this report. Major contributors to
this report were John E. Clary, Ronald J. Salo, James B. Smoak, and
Ralph O. White.
Sincerely yours,
William T. Woods:
Director:
Acquisition and Sourcing Management:
Signed by William T. Woods:
Appendix:
Comments From the General Services Administration:
GSA:
GSA Administrator:
February 5, 2003:
The Honorable David M. Walker Comptroller General:
of the United States General Accounting Office Washington, DC 20548:
Dear Mr. Walker:
Thank you for providing us with the opportunity to comment on the
General Accounting Office (GAO) draft report entitled ’Electronic
Procurement: Business Strategy Needed for GSA Advantage Program“ (GAO-
03-328), dated January 16, 2003. The draft recommends that the General
Services Administration (GSA) develop a comprehensive business strategy
that identifies GSA‘s objectives for an on-line market research and
ordering system, the desired capabilities of that system, the resources
required to implement the system, and measures to track performance.
GSA concurs with the draft report‘s recommendation.
GSA‘s Federal Supply Service (FSS) is currently conducting an FSS-wide
enterprise IT architecture study during FY 2003. GSA recognizes that it
is time to perform a rigorous analysis that will document the business
case for our future decisions, and in fact, we are in the process of
developing a strategic business plan for GSA Advantage as part of that
effort.
If you have any additional questions or need further assistance, please
have a member of your staff contact Mr. Ralph Boldt, Branch Chief,
Audit Followup and Evaluation Branch, on (202) 501-3094.
Sincerely,
Stephen A. Perry, Administrator:
Signed by Stephen A. Perry:
cc: William T. Woods, Director:
Acquisition and Sourcing Management United States General Accounting
Office Washington, DC 20548:
U.S. General Services Administration 1800 F Street, NW:
Washington, DC 20405-0002 Telephone: (202) 501-0800 Fax: (202) 219-1243
www.gsa.gov:
FOOTNOTES
[1] The Clinger-Cohen Act called for the economic and efficient
procurement of information technology and other commercial items using
a system that provides on-line access to price, features, and
performance information on all products and services found in FSS
schedules, and that provides on-line ordering capability. Although the
system continues, the statutory requirement for the system was repealed
by Section 825(c)(1) of the Bob Stump National Defense Authorization
Act for Fiscal Year 2003, Pub. L. No. 107-314, 116 Stat. 2616.
[2] This calculation does not include holidays and weekends.
[3] Advantage sales also include purchases of stock items from GSA‘s
two remaining warehouses. GSA‘s stock program that comprises about 55
percent of the Advantage sales (in fiscal year 2002) has had a 4-year
decline in sales overall due to a changing customer base and the
increased use of credit cards throughout the government. In fact, FSS
has closed all but two warehouses and transitioned from a government-
owned and operated supply distribution network to a system of alliances
with private companies to provide the goods and services. Without
alternative strategies such as a virtual store through Advantage that
would allow GSA to provide the products to the agencies using direct
delivery rather storing and shipping them from a warehouse, agency
officials told us the stock program sales might continue to decline.
[4] Loaded labor rates include salary, fringe benefits, overhead, and
profit.
[5] See U.S. General Accounting Office, Contract Management: Not
Following Procedures Undermines Best Pricing Under GSA‘s Schedule, GAO-
01-125 (Washington D.C.: Nov. 28, 2000).
GAO‘s Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO‘s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO‘s Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ’Today‘s Reports,“ on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select ’Subscribe to daily E-mail alert for newly
released products“ under the GAO Reports heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
