Payment Processing
Statistical Sampling Plan for Voucher Prepayment Examination Gao ID: GAO-03-1067R August 28, 2003This report responds to a request made by the Defense Finance and Accounting Service (DFAS) that we approve the use of a statistical sampling plan for voucher prepayment examination of contract payments. While Title VII, "Fiscal Guidance," of the GAO Policy and Procedures Manual for Guidance of Federal Agencies requires prepayment examination of vouchers prior to their certification and payment, it permits the use of statistical sampling for vouchers up to $2,500. DFAS requested a waiver of the $2,500 ceiling for statistical sampling and approval of an alternative voucher prepayment examination plan that would, among other things, use statistical sampling for certain vouchers up to a maximum of $500,000 for contract payments.
At this time, we cannot approve the request for a waiver or the proposed alternative sampling plan. The submission, requesting exception to the $2,500 maximum for statistical sampling in voucher prepayment examination, does not justify the projected savings of approximately $4.2 million per year. Further, it is unclear how the proposed plan will adequately protect the government's interest. In addition, absent GAO approval, DFAS certifying officers risk personal liability for improper payments should they rely upon the voucher prepayment examination process under the proposed alternative sampling plan.
GAO-03-1067R, Payment Processing: Statistical Sampling Plan for Voucher Prepayment Examination
This is the accessible text file for GAO report number GAO-03-1067R
entitled 'Payment Processing: Statistical Sampling Plan for Voucher
Prepayment Examination' which was released on August 28, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
On January 5, 2004, this document was revised to add various
footnote references missing in the text of the body of the document.
August 28, 2003:
Ms. Martha Stearns:
Deputy Director:
Commercial Pay Services:
Defense Finance and Accounting Service:
P.O. Box 182317:
Columbus, Ohio 43218-2317:
Subject: Payment Processing: Statistical Sampling Plan for Voucher
Prepayment:
Examination:
Dear Ms. Stearns:
This report responds to a request made by your office that we approve
the use of a statistical sampling plan for voucher prepayment
examination of contract payments. While Title VII,[Footnote 1] "Fiscal
Guidance," of the GAO Policy and Procedures Manual for Guidance of
Federal Agencies requires prepayment examination of vouchers prior to
their certification and payment, it permits the use of statistical
sampling for vouchers up to $2,500. You requested a waiver of the
$2,500 ceiling for statistical sampling and approval of an alternative
voucher prepayment examination plan that would, among other things, use
statistical sampling for certain vouchers up to a maximum of $500,000
for contract payments.
In order to respond to your request, we reviewed your letter and the
information provided in the attachments. We also contacted your staff
to discuss your request in more detail. We analyzed the sections of the
law pertaining to this request and reviewed the legislative history of
31 U.S.C. § 3521(b), upon which Title VII is based. Section 3521(b) of
Title 31 provides for statistical sampling procedures. We did not test
the Defense Finance and Accounting Service (DFAS) contract payment
system; therefore, our response only addresses your request
conceptually.
At this time, we cannot approve your request for a waiver or your
proposed alternative sampling plan. Your submission, requesting
exception to the $2,500 maximum for statistical sampling in voucher
prepayment examination, does not justify your projected savings of
approximately $4.2 million per year. Further, it is unclear how the
proposed plan will adequately protect the government's interest. In
addition, absent GAO approval, DFAS certifying officers risk personal
liability for improper payments should they rely upon the voucher
prepayment examination process under the proposed alternative sampling
plan.
Background:
As described in your letter and the information accompanying it and as
explained by your staff during subsequent discussions, DFAS is
evaluating its existing contract payment procedures with the intent of
achieving cost-effective compliance with Title VII. We support such
initiatives to create a government that works better and costs less. At
the same time, agencies have the responsibility to protect the
government's interest.
The fiscal guidance in Title VII contains general accounting processing
procedures that are required by law. The requirements are designed to
complement agencies' existing federal accounting, internal control, and
systems standards. All basic vouchers, voucher schedules, and invoices
or bills used as vouchers must be certified for correctness and
legality by an authorized certifying officer. A certifying officer is
accountable for and required to personally reimburse the government for
any illegal, improper, or incorrect payment made by a disbursing
officer because of his or her certification.[Footnote 2] In addition, a
disbursing official is accountable for ensuring that a voucher is in
proper form, certified and approved, and computed correctly based on
the facts certified.[Footnote 3] In appropriate circumstances, the
Comptroller General may relieve a disbursing or certifying official
from liability.[Footnote 4]
Electronic techniques and systems are being used to generate, process,
transmit, and store financial information in today's federal financial
environment, requiring accountable officials to rely on the systems,
controls, and personnel that process the transactions. However, the
accountable officials must have reason to believe that the key
processes and controls on which they rely are working.[Footnote 5]
Further, even though all or some portion of the payment process may be
automated, the certifying officer's basic accountability and legal
liability for the propriety of the payment remains unaltered.
As a viable alternative to the traditional practice of performing 100
percent prepayment examination of vouchers, section 7.4E of Title VII
authorizes agencies to establish statistical sampling programs for the
examination of vouchers in support of their certification and payment,
provided the vouchers do not exceed $2,500. Within this maximum,
agencies are required to establish their own dollar limitations based
on cost-benefit analyses of their voucher examining
operations.[Footnote 6] Any disbursing or certifying officer relying in
good faith on the statistical sampling procedure adopted by the agency
to disburse funds or certify a voucher for payment will not be liable
for losses to the government resulting from payment or certification of
a voucher not audited specifically because of the use of an authorized
sampling procedure, provided that the agency has diligently carried out
collection actions prescribed by the Comptroller General.[Footnote 7]
Exceptions to the $2,500 maximum for vouchers that may be statistically
sampled may be authorized by the Comptroller General, on a case-by-case
basis, when it is clear that they are warranted economically and that
the procedures and controls will adequately protect the government's
interest.
Prepayment Examination Process at DFAS:
DFAS's voucher prepayment examination process is designed to ensure,
among other requirements, that (1) payment is permitted by law, (2)
appropriation amounts are available at the time and are being used for
the intended purpose, (3) goods and services have been received and
conform to the requirements of the order or agreement, (4) required
administrative approvals have been obtained, and (5) quantities,
prices, and calculations are accurate.
According to the information submitted with your request, the DFAS
Columbus Contract Pay organization entitles[Footnote 8] about 78,000
vouchers monthly at an average total monthly voucher value of $7.2
billion. Contractor invoices are processed and paid through a contract
administration and payment system--Mechanization of Contract
Administrative Services (MOCAS). Approximately 35,000 of the 78,000
monthly invoices are entitled through an automated payment of invoices
(API) process. In the API process, the MOCAS system edit checks provide
for the entitlement and automatic payment of these invoices. The
remaining 43,000 payment vouchers are manually entitled by
approximately 120 voucher examiners and supervisors. Vouchers are
manually entitled if, for example, the contract governing the voucher
payment has been modified, the contract has special payment
instructions, or it fails the edits in the API process.
The DFAS Front End Analysis (FEA) organization was established to
detect contract pay voucher errors prior to disbursement. One primary
function of the DFAS FEA organization is to perform independent voucher
prepayment reviews. FEA staff judgmentally select vouchers at various
points in the entitlement process and examine them for various
attributes, such as input errors, payment according to contract, and
availability of funds. According to your letter, FEA's voucher
examination procedures are designed to assess MOCAS system errors or
individual voucher examiner error rates by nonstatistical sampling from
each day's manual entitlements.
Of the approximately 78,000 vouchers processed each month, 55,000 are
greater than $2,500 and therefore subject to 100 percent prepayment
examination in accordance with Title VII. Currently, FEA staff
nonstatistically select and examine an average of 1,500 vouchers above
$2,500 per month. In a sample of manually entitled invoices from
October 2000 through July 2001, FEA noted a primary error rate[Footnote
9] of 13.4 percent, with some of those errors attributable to
overpayments totaling approximately $4.8 million for the 10-month
period.
DFAS's Proposed Alternative Sampling Plan for Prepayment Examination:
Your alternative voucher prepayment examination plan for contract
payments would shift the role of the FEA independent voucher team, made
up of 6.5 full-time equivalents, from detecting input errors and
detecting individual voucher examiner errors to playing a key role in
the prepayment certification process for accountable officers.
Specifically, FEA staff would perform a review of (1) 100 percent of
all manually entitled payment vouchers equaling or exceeding $500,000
and (2) a statistical sample of manually entitled vouchers less than
$500,000 combined with all API vouchers regardless of amount, thereby
excluding API vouchers from the 100 percent voucher examination
threshold. All vouchers selected as part of this sampling plan would be
reviewed to ensure that they were legal, proper, and correct for
payment using standardized review criteria. Any discrepancies noted
would be reported, tracked, monitored, and resolved.
Your analysis noted that this more systematic examination of vouchers
would ensure that FEA staff review over 53 percent of total monthly
contract payment dollars, as compared to the 6.5 percent of the total
dollars that are currently examined through the judgmental sampling of
manually entitled vouchers only. In turn, DFAS expects that this
heightened review will detect and correct more potential overpayments
and result in savings to the government. The analysis prepared by a
contractor for DFAS concludes that the 6.5 FEA full-time equivalents
can perform 1,000 manually entitled voucher examinations per month if
the maximum threshold for statistically sampled vouchers was raised
from $2,500 to $500,000. Thus, it appears to us that the resources
available are driving the proposed threshold and sample size rather
than the consideration of other key factors, such as an appropriate
risk level and acceptable error rate that would provide reasonable
assurance that the government's interest is protected.
Our Assessment of the Proposed Alternative Sampling Plan:
Your proposed alternative sampling plan, which includes waiving the
$2,500 maximum for statistical sampling of voucher prepayment
examination for contract payments, does not justify the proposed
savings or provide assurance that the government's interest will be
adequately protected. Further, DFAS certifying officers risk personal
liability for improper payments should they rely upon the prepayment
examination process under the proposed alternative sampling plan.
GAO's Policy and Procedures Manual sets out a relatively
straightforward formula for agencies to use in deciding whether to
adopt sampling in the certification of payment vouchers under 31 U.S.C.
§ 3521(b). The cost of examining all vouchers should be compared with
the total costs of sampling (i.e., the cost of examining the sample
plus the amount of projected losses from undetected errors in vouchers
not examined).[Footnote 10] Where the costs of reviewing all vouchers
exceed the costs of sampling plus estimated losses, sampling offers
savings. Savings are achieved when the combined costs of (1) examining
the sample and (2) projected losses due to undetected errors on
invoices not examined are less than the cost of examining all invoices.
Through analysis, the sampling plan must identify a tolerable error
rate,[Footnote 11] the number of invoices to select for examination,
and the selection method.
Your request describes approximately $4.2 million in projected savings
per year as a result of the alternative sampling plan. Based on our
review of your proposed sampling plan, we question this projection.
Specifically, adequate evaluation of sample results involves defensible
projections to the entire population using statistically valid samples.
The sample results in your plan cannot be projected to the population
because it is not clear how the monthly sample sizes were determined or
how the samples were picked from the monthly population. The DFAS
analysis is centered on 10 monthly selected samples of manually
entitled vouchers, but does not document how the sample sizes, ranging
from 985 vouchers selected in July 2001 to 1,894 vouchers selected in
April 2001, were determined relative to the total monthly population of
manually entitled vouchers. In addition, the methodology used to record
errors in the sample is unclear in the analysis. The analysis indicates
that the error rate of 13.4 percent is a result of 2,019 "primary"
errors noted in the 15,021 vouchers reviewed. A primary error is
defined in the analysis as "the first error on a reviewed invoice."
However, an invoice may have more than one error. Because total errors
are not known or identified, the results cannot be viewed as
representative and used to project potential losses to the remaining
vouchers in the population. Hence the measured savings calculated as a
result of FEA's prospective ability to detect and correct potential
contract payment errors under the proposed plan are unjustified.
We have additional concerns about whether this plan adequately protects
the government's interest, based on the Department of Defense's (DOD)
long-standing problems with accurately accounting for and reporting on
disbursements. We have previously reported[Footnote 12] on long-
standing problems in MOCAS contract payments. For example, recent DFAS
data showed that almost $1 of every $4 in contract payment transactions
was for adjustments to previously recorded payments--$49 billion of
adjustments out of $198 billion in transactions. DOD had been
developing a new system, the Defense Payment Procurement System (DPPS),
which was designed in part to resolve DOD's long-standing disbursements
problems and streamline contract payment processes. However, as we
previously reported,[Footnote 13] DOD terminated DPPS in December 2002
after 7 years of development due to poor program performance and
increasing life cycle costs. Consequently, DFAS will be relying on
MOCAS, which was implemented in 1968, and other legacy systems to meet
its requirements for the foreseeable future.[Footnote 14] In addition,
DOD contracts processed by DFAS contain numerous fund citations and
complex payment allocation terms. These factors, along with contract
modifications, changes in payment allocation terms, and nonstandardized
contracts and payment terms contribute to the fact that a large
percentage of payment vouchers will need to be manually entitled for
the foreseeable future. Based on our prior findings, this area has been
very susceptible to errors.
One of the requirements imposed on the certifying officer is to ensure
(or have in place a system that can offer a reasonable degree of
assurance of) the "legality of a proposed payment under the
appropriation or fund involved."[Footnote 15] A certifying officer's
legal liability is strict and arises automatically at the time of an
illegal or improper payment.[Footnote 16] The Comptroller General has
authority to relieve a certifying officer from liability when we find
that the certification was based on official records and the official
did not know, and by reasonable diligence and inquiry could not have
discovered, the correct information.[Footnote 17]
The Federal Managers' Financial Integrity Act of 1982 (FMFIA) requires
agency heads to periodically evaluate their systems of internal control
using the guidance issued by the Office of Management and Budget
(OMB).[Footnote 18] Title VII also provides that FMFIA reviews should
be designed, carried out, and reported in ways that show whether the
voucher processing and disbursing system and controls are reliable. To
the extent practicable, the needs of certifying and approving officials
should be considered in advance of an FMFIA review, and the review
results should be made available or communicated to them in reports
tailored to their special interests. The FMFIA review is a good
starting place for assessing risk in the contract pay certification and
disbursing process, which, used in conjunction with an objective and
scientifically developed approach encompassing the criteria[Footnote
19] referred to above, can provide the guidance for alternative plans
that we could evaluate in the future.
Conclusion:
Title VII allows for a deviation from the $2,500 limitation for using
statistical sampling to certify vouchers to the extent that there are
resultant savings while adequately protecting the government's
interest. Title VII does not distinguish between certification of
automated or manual entitlements, but provides for statistical sampling
of vouchers less than $2,500 as a technique in large-volume, highly
automated voucher processing operations. Any alternatives to 100
percent prepayment examination must be part of a total system that
includes well-defined responsibilities, organizational structures, and
an adequate set of internal controls. Until a sound statistical
sampling plan with justifiable savings is presented, we cannot grant a
waiver of the $2,500 maximum for statistical sampling of contract
payments or approve the alternative voucher examination plan submitted.
The proposed alternative sampling plan does not justify savings or
adequately protect the government's interest. In addition, good faith
reliance on this particular plan would not relieve a disbursing or
certifying official of liability for an improper payment or
certification of a voucher not audited.
The contents of this report were discussed with Bill Felice of your
staff. We hope our comments are helpful. If you have any questions, or
would like to discuss these matters further, please contact me at (202)
512-6906 or Mary Arnold Mohiyuddin, Assistant Director, at (202) 512-
3087. This report will be available at no charge on GAO's Web site at
http://www.gao.gov.
Sincerely yours,
McCoy Williams:
Director:
Financial Management and Assurance:
Signed by McCoy Williams:
(190060):
FOOTNOTES
[1] Title VII of GAO's Policy and Procedures Manual for Guidance of
Federal Agencies. The manual is the official medium through which the
Comptroller General promulgates (1) accounting principles, standards,
and related requirements and material for the guidance of federal
agencies in the development of their accounting systems and internal
auditing programs, (2) uniform procedures for use by federal agencies,
and (3) regulations governing GAO's relationships with other federal
agencies and with private concerns doing business with the government.
31 U.S.C. § 3521(b) states that heads of agencies may prescribe
statistical sampling procedures to audit vouchers when they decide that
economies will result. The term audit as used in 31 U.S.C. § 3521 (b)
refers to the examination of disbursement vouchers prior to their
certification for payment.
[2] 31 U.S.C. § 3528 (a)(4).
[3] 31 U.S.C. § 3325 (a)(2).
[4] 31 U.S.C. § 3528 (b).
[5] The most significant factors on which confidence in the system and
controls should be based are (1) a well-defined organization structure
and flow of work, with appropriate separation of responsibilities and
clearly written policies and procedures governing the examination,
approval, and certification of disbursement vouchers, (2) effective
application of available technology and concepts to achieve efficient
and effective voucher processing, and (3) review of the voucher
processing procedures and controls in sufficient scope, depth, and
frequency to provide reasonable assurance that key processing
procedures and controls are working and reliable.
[6] GAO's Policy and Procedures Manual for Guidance of Federal
Agencies, Title VII, "Fiscal Procedures," ch. 7, sec. 7.4E.
[7] 31 U.S.C. § 3521 (c).
[8] The entitlement process, according to DFAS, refers to the
preparation of the payment package, including the matching of an
invoice, receiving report, and contract. Entitlement precedes
prepayment review procedures.
[9] A primary error is defined in the business case analysis as "the
first error on an invoice that has been reviewed." An invoice may have
more than one error.
[10] GAO's Policy and Procedures Manual for Guidance of Federal
Agencies, Title VII, app. III.
[11] Tolerable error rate is the point at or below which savings occur.
[12] U.S. General Accounting Office, Canceled DOD Appropriations:
$615 Million of Illegal or Otherwise Improper Adjustments, GAO-01-697
(Washington, D.C.: July 26, 2001).
[13] U.S. General Accounting Office, DOD Business Systems
Modernization: Continued Investment in Key Accounting Systems Needs to
Be Justified, GAO-03-465 (Washington, D.C.: Mar. 28, 2003).
[14] U.S. General Accounting Office, DOD Financial Management:
Integrated Approach, Accountability, Transparency, and Incentives Are
Keys to Effective Reform, GAO-02-573T (Washington, D.C.: Mar. 20,
2002).
[15] 31 U.S.C. § 3528(a)(3).
[16] 54 Comp. Gen. 112, 114 (1974).
[17] 31 U.S.C. § 3528 (b)(1).
[18] 31 U.S.C. § 3512 (d).
[19] The criteria include (1) defining the universe of vouchers, (2)
determining the size of the sample needed, and (2) analyzing the
results of the sample using the appropriate statistical procedure.
