Tax Debt Collection
IRS Could Improve Future Studies by Establishing Appropriate Guidance Gao ID: GAO-10-963 September 24, 2010In September 2006, the Internal Revenue Service (IRS) started the private debt collection (PDC) program for using private collection agencies (PCA) to help collect some unpaid tax debts. Aware of concerns that PCAs might cost more than using IRS staff, IRS began studying the collection costs and performance of PCAs and IRS. In March 2009, IRS announced that it would not renew its PCA contracts based on the study and announced plans for increasing collection staffing. As requested, GAO is reporting on whether (1) the study was sound as primary support for IRS's PDC decision and (2) IRS has planned or made changes to its collection approach based on its PCA experience and PDC study. GAO compared IRS's study to federal and other guidance on what should be included in analyses to support program decisions and analyzed IRS's changes given expectations that IRS would consider PCAs' best practices.
IRS's comparative study of the PDC program was not soundly designed to support its decision on whether to continue contracting out debt collection. Although the study was not originally intended or designed as primary support for the decision, IRS officials nonetheless used it as such. IRS did not have guidance for program managers on the type of analysis that should be done to support decisions to create, renew, or expand programs. IRS had not retained sufficient documentation on the sample used in the study or documented some analyses that would have been helpful if performed. The study results may be overstated or understated because the study sample was not generalizable to the program as a whole. The study had a narrow objective of comparing results for IRS working the same cases as PCAs had, and as a result, the study design did not consider other factors recommended by Office of Management and Budget and other guidance on conducting program analysis. For example, the study did not analyze alternatives to program scale, such as expanding it or scaling it back. Program analysis guidance states that to the extent possible all costs and benefits should be counted and alternative means of achieving a program's goals should be considered. But the study did not identify important costs and benefits, such as whether taxpayers' compliance costs would be different if IRS or PCAs work debt cases. Nevertheless, neither GAO nor IRS officials know whether the study results and decision on the program would have differed significantly if it had been designed to be primary support for IRS's PDC program. In commenting on a draft of GAO's report, IRS disagreed that the PDC study was not soundly designed. GAO stands by its analysis detailing the study's errors, narrow scope, and lack of adherence to guidance. These design and methodology deficiencies limited the study's usefulness in supporting IRS's decision. IRS has not made or planned changes to its collection approach based on its PCA experience and study. In authorizing the use of PCAs, Congress required IRS to report to Congress its measurement plan to identify any of the PCAs' best practices that IRS could adopt to improve its own collection operations. IRS did not continue to report to Congress as required. In an unpublished draft report, IRS asserted that it had reviewed a number of PCA practices and found no immediate opportunities to change its collection approach. IRS did not provide GAO documentation on the study to support that conclusion. In part because PCA-type cases had previously been considered low priority, IRS officials were surprised by the PDC study results, which indicated that IRS staff might have better results working PCA-type cases than some of the cases IRS normally works. IRS officials said that they initiated a pilot study in 2009 to help them decide whether to use IRS staff to work selected types of PCA cases. As GAO concluded its review, IRS provided conflicting information on the role of the pilot study. On one hand, IRS said a collection selection system to be implemented in January 2011 overtook the need for the study. On the other hand, an IRS official said that the results from PCA-type cases were not used in the development of the new case selection system. GAO recommends that IRS (1) establish guidance on analyses to support program decisions, (2) establish a policy requiring documentation of program studies, and (3) ensure that PCA-type case results are considered for IRS's new case selection model. IRS agreed with the first two recommendations and agreed in principle with the third, which GAO revised to reflect updated information that IRS provided.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Michael Brostek Team: Government Accountability Office: Strategic Issues Phone: (202) 512-9039GAO-10-963, Tax Debt Collection: IRS Could Improve Future Studies by Establishing Appropriate Guidance
This is the accessible text file for GAO report number GAO-10-963
entitled 'Tax Debt Collection: IRS Could Improve Future Studies by
Establishing Appropriate Guidance' which was released on October 26, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Ranking Member, Committee on Finance, U.S. Senate:
United States Government Accountability Office:
GAO:
September 2010:
Tax Debt Collection:
IRS Could Improve Future Studies by Establishing Appropriate Guidance:
GAO-10-963:
GAO Highlights:
Highlights of GAO-10-963, a report to the Ranking Member, Committee on
Finance, U.S. Senate.
Why GAO Did This Study:
In September 2006, the Internal Revenue Service (IRS) started the
private debt collection (PDC) program for using private collection
agencies (PCA) to help collect some unpaid tax debts. Aware of
concerns that PCAs might cost more than using IRS staff, IRS began
studying the collection costs and performance of PCAs and IRS. In
March 2009, IRS announced that it would not renew its PCA contracts
based on the study and announced plans for increasing collection
staffing. As requested, GAO is reporting on whether (1) the study was
sound as primary support for IRS‘s PDC decision and (2) IRS has
planned or made changes to its collection approach based on its PCA
experience and PDC study. GAO compared IRS‘s study to federal and
other guidance on what should be included in analyses to support
program decisions and analyzed IRS‘s changes given expectations that
IRS would consider PCAs‘ best practices.
What GAO Found:
IRS‘s comparative study of the PDC program was not soundly designed to
support its decision on whether to continue contracting out debt
collection. Although the study was not originally intended or designed
as primary support for the decision, IRS officials nonetheless used it
as such. IRS did not have guidance for program managers on the type of
analysis that should be done to support decisions to create, renew, or
expand programs. IRS had not retained sufficient documentation on the
sample used in the study or documented some analyses that would have
been helpful if performed. The study results may be overstated or
understated because the study sample was not generalizable to the
program as a whole. The study had a narrow objective of comparing
results for IRS working the same cases as PCAs had, and as a result,
the study design did not consider other factors recommended by Office
of Management and Budget and other guidance on conducting program
analysis. For example, the study did not analyze alternatives to
program scale, such as expanding it or scaling it back. Program
analysis guidance states that to the extent possible all costs and
benefits should be counted and alternative means of achieving a program‘
s goals should be considered. But the study did not identify important
costs and benefits, such as whether taxpayers‘ compliance costs would
be different if IRS or PCAs work debt cases. Nevertheless, neither GAO
nor IRS officials know whether the study results and decision on the
program would have differed significantly if it had been designed to
be primary support for IRS‘s PDC program. In commenting on a draft of
GAO‘s report, IRS disagreed that the PDC study was not soundly
designed. GAO stands by its analysis detailing the study‘s errors,
narrow scope, and lack of adherence to guidance. These design and
methodology deficiencies limited the study‘s usefulness in supporting
IRS‘s decision.
IRS has not made or planned changes to its collection approach based
on its PCA experience and study. In authorizing the use of PCAs,
Congress required IRS to report to Congress its measurement plan to
identify any of the PCAs‘ best practices that IRS could adopt to
improve its own collection operations. IRS did not continue to report
to Congress as required. In an unpublished draft report, IRS asserted
that it had reviewed a number of PCA practices and found no immediate
opportunities to change its collection approach. IRS did not provide
GAO documentation on the study to support that conclusion. In part
because PCA-type cases had previously been considered low priority,
IRS officials were surprised by the PDC study results, which indicated
that IRS staff might have better results working PCA-type cases than
some of the cases IRS normally works. IRS officials said that they
initiated a pilot study in 2009 to help them decide whether to use IRS
staff to work selected types of PCA cases. As GAO concluded its
review, IRS provided conflicting information on the role of the pilot
study. On one hand, IRS said a collection selection system to be
implemented in January 2011 overtook the need for the study. On the
other hand, an IRS official said that the results from PCA-type cases
were not used in the development of the new case selection system.
What GAO Recommends:
GAO recommends that IRS (1) establish guidance on analyses to support
program decisions, (2) establish a policy requiring documentation of
program studies, and (3) ensure that PCA-type case results are
considered for IRS‘s new case selection model. IRS agreed with the
first two recommendations and agreed in principle with the third,
which GAO revised to reflect updated information that IRS provided.
View [hyperlink, http://www.gao.gov/products/GAO-10-963] or key
components. For more information, contact Michael Brostek at (202) 512-
9110 or brostekm@gao.gov.
[End of section]
Contents:
Letter:
Scope and Methodology:
Background:
Methodological Errors and Narrow Scope Limit the Study's Usefulness in
Supporting the PDC Decision:
IRS Has Not Changed Its Collection Approach Based on Its PDC
Experience and Study, and Whether IRS Will Work Certain PCA-Type Cases
Is Unclear:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Comments from the Internal Revenue Service:
Appendix II: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: IRS's PDC Study Results Comparing IRS and PCAs Working
Collection Inventory from the Same Pool of Cases Assigned to PCAs:
Table 2: Summary of the Types of Costs Included in IRS's PDC Study:
Abbreviations:
ACS: Automated Collection System:
CDA: Consolidated Decision Analytics:
IRM: Internal Revenue Manual:
IRS: Internal Revenue Service:
OMB: Office of Management and Budget:
OPERA: Office of Program Evaluation and Risk Analysis:
PCA: private collection agency:
PDC: private debt collection:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
September 24, 2010:
The Honorable Charles E. Grassley:
Ranking Member:
Committee on Finance:
United States Senate:
Dear Senator Grassley:
In October 2004, Congress authorized the Internal Revenue Service
(IRS) to create the private debt collection (PDC) program to collect
some portion of the tax debt inventory. The program enabled IRS to
collect tax debts by contracting with private collection agencies
(PCA) and paying them out of a revolving fund of the revenue
collected.[Footnote 1] Some in Congress and elsewhere in the tax
community, including the National Taxpayer Advocate, expressed
concerns that using PCAs might result in higher costs as compared to
increasing IRS resources to collect the debts.[Footnote 2] Aware of
these concerns, IRS officials said that if Congress authorized the use
of PCAs, IRS would study the comparative performance of PCAs and IRS
in collecting unpaid taxes. The law authorizing the program also
required that IRS create a measurement plan to identify and capture
information on the best practices used by PCAs compared with IRS's
practices to identify any that IRS could adopt to improve its own
collection operations.[Footnote 3] IRS began assigning cases to PCAs
and began the PDC study in September 2006.
In March 2009, IRS released the PDC study and announced that IRS would
not renew expiring contracts with PCAs. The announcement stated that
IRS had reviewed the PDC program and specifically cited the results of
the study, an independent review of it, and its conclusion that IRS is
more cost effective than PCAs when working similar inventory.[Footnote
4] The announcement also noted that IRS anticipated hiring additional
collection staff in fiscal year 2009 and quoted the Commissioner of
Internal Revenue as saying, "I believe this work is best done by IRS
employees, and I believe we have strong support from the
Administration and the Congress for increased IRS enforcement
resources going forward."
You asked us to review the soundness of IRS's PDC study and address
questions related to changes IRS made since it stopped using PCAs.
Specifically, our objectives were to determine the following:
* How sound was IRS's PDC study as primary support for IRS's decision
to discontinue contracting out tax debt collection?
* What changes has IRS planned or made to its collection approach
based on its PCA experience and the PDC study?
Scope and Methodology:
To determine the soundness of IRS's PDC study as primary support for
IRS's decision to discontinue contracting out tax debt collection, we
reviewed the study report and supporting documents and other data. We
interviewed IRS officials and contractors involved in the study. We
also reviewed the report IRS commissioned to validate the PDC study
conclusion and interviewed contractor staff involved in that effort.
[Footnote 5] We also interviewed officials from one of the PCA firms
IRS contracted with to obtain their views on the PDC study.[Footnote
6] We reviewed and summarized program analysis guidance from various
sources in developing our criteria. Such guidance came from Office of
Management and Budget (OMB) Circular A-94,[Footnote 7] a previous GAO
publication on evaluating federal programs,[Footnote 8] an academic
research paper on costs and benefits that should be considered in
making decisions on resources for tax enforcement programs,[Footnote
9] and accepted quantitative analysis criteria on sampling cases and
projecting results. We compared IRS's study methodology and report to
the criteria from the various guidance. We also reviewed IRS's
Internal Revenue Manual (IRM) and interviewed officials to determine
if IRS had guidance on whether and how to conduct and document
economic analyses to support decisions to initiate, renew, or expand
programs. We also reviewed the types of costs and benefits that IRS
included in the PDC study.
To determine what changes IRS has planned or made to its collection
approach based on its PCA experience and the PDC study, we reviewed
program documents and interviewed IRS officials on their processes and
procedures for collecting tax debt. We also compared IRS's plans for
studying whether to work on PCA-type cases in the future to guidance
in our previous publication on evaluating federal programs[Footnote
10] and in our report reviewing IRS's study of the earned income tax
credit.[Footnote 11]
We conducted this performance audit from November 2009 to September
2010 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Background:
In developing the PDC program, IRS officials said that contracting
with PCAs was needed because Congress was unlikely to provide IRS
sufficient staff to attempt collection on the inventory of cases with
a lower priority.[Footnote 12] Also, officials said that PCAs would
pursue cases that IRS staff would not because IRS had other higher-
priority collection cases.
As IRS planned the PDC program, we issued two reports with findings
and recommendations to improve the program.
* In May 2004, we reported that IRS's PDC study approach--comparing
PCA and IRS performance for the same type of simpler cases that would
be sent to PCAs--would provide limited information to judge whether
using PCAs is the best use of resources.[Footnote 13] In sum, the
approach conflicted with IRS officials' position that these simpler
cases would not be worked by IRS employees given the higher-priority
cases in IRS's workload. We recommended instead that IRS compare the
use of PCAs to a collection strategy that officials determine to be
the most effective and efficient overall way of achieving collection
goals.
* In September 2006, we reported that IRS's planned PDC study approach
would not meet the intent of our 2004 recommendation.[Footnote 14]
Because the study would not count the fees paid to PCAs as program
costs, it would not compare the results of using PCAs with the results
IRS could get if it was given the same amount of resources, including
the fees to be paid to the PCAs. We recommended instead that IRS
ensure that the study methodology and the reports on the study include
the full costs of the PDC program, including the fees paid to PCAs and
the best use of those federal funds.
IRS agreed with the recommendations in both reports, and IRS's actions
to implement them generally are the topic of this report. IRS's PDC
study did not meet the intent of our 2004 recommendation because it
did not compare using PCAs to what IRS officials determined to be the
most efficient and effective overall strategy. Although IRS met part
of our 2006 recommendation in that the PDC study included fees paid to
PCAs as program costs, the PDC study did not fully implement our
recommendation to include the best use of federal funds because, as
discussed further below, the study had methodological errors and a
narrow scope.
IRS's PDC study report results were released on March 5, 2009, along
with IRS's announcement that it was ending the program.[Footnote 15]
The study tracked selected cases assigned to IRS's Automated
Collection System (ACS) versus PCAs and measured them in terms of cost
per dollar collected, percentage of balance due collected, and
percentage of cases in payment status at the end of the study period.
[Footnote 16] For both PCAs and IRS, the PDC study counted the dollars
collected from full payments or estimated dollars collected through
installment agreements. The study results, summarized in table 1,
showed that IRS performed better in each of the measures.
Table 1: IRS's PDC Study Results Comparing IRS and PCAs Working
Collection Inventory from the Same Pool of Cases Assigned to PCAs:
IRS:
Dollars collected (estimated): $775,302;
Cost: $53,545;
Cost per dollar collected: $0.07;
Balance due: $6.8 million;
Percentage of balance due collected: 11;
Modules[A]: assigned to be worked: 1,341;
Modules[A] in payment status: 369;
Percentage of modules[A] in payment status: 28.
PCAs:
Dollars collected (estimated): $443,438;
Cost: $105,621;
Cost per dollar collected: $0.24;
Balance due: $11.7 million;
Percentage of balance due collected: 4;
Modules[A]: assigned to be worked: 2,133;
Modules[A] in payment status: 284;
Percentage of modules[A] in payment status: 13.
Source: IRS.
[A] A module in this case is a tax year for which there was an amount
of unpaid tax due from a taxpayer. Quarterly employment taxes were not
included.
[End of table]
On February 3, 2009, legislation was introduced in the House to amend
the Internal Revenue Code of 1986 to repeal the authority of the
Secretary of the Treasury to enter into PDC contracts.[Footnote 17]
The last major action on the legislation was on February 3, 2009, when
it was referred to the House Committee on Ways and Means.
The Omnibus Appropriations Act of 2009, enacted March 11, 2009,
prevented IRS from using fiscal year 2009 appropriated funds "to enter
into, renew, extend, administer, implement, enforce, or provide
oversight of any qualified tax collection contract (as defined in
section 6306 of the Internal Revenue Code of 1986)."[Footnote 18]
Absent the access to appropriated funds, IRS funded the administrative
costs of the PDC program through its user fees[Footnote 19] until IRS
was able to end all PDC activities. The Consolidated Appropriations
Act of 2010 placed the same prohibition on IRS's use of fiscal year
2010 funds.[Footnote 20]
Methodological Errors and Narrow Scope Limit the Study's Usefulness in
Supporting the PDC Decision:
According to IRS officials, the PDC study was not originally intended
or designed to be primary support for IRS's PDC program decision. Even
though other factors, such as potential increases in IRS collection
staffing, were considered, based on our interviews with IRS officials
and IRS's announcement of the program's termination, the study results
played a primary role in supporting the decision. IRS officials noted
that the difference in cost per dollar collected using IRS staff and
PCAs was so pronounced that in their view, additional analyses would
have been unlikely to change the decision that was made. Nevertheless,
neither we nor IRS officials know whether the PDC study results and
decision on the program would have differed significantly if the study
had been designed to be primary support for IRS's PDC program.
However, errors in the study sampling methodology and the study's
narrow scope limit its usefulness in supporting the PDC decision.
IRS does not have guidance on whether and how to conduct and document
economic analyses to support decisions to initiate, renew, or expand
programs. The IRM, which includes authoritative guidance for IRS
managers, does not include guidance on economic analyses that should
support decisions to initiate, renew, or expand a program.
The Study Result May Be Over-or Understated Because the Sample Was Not
Generalizable to the PDC Population:
Because IRS did not perform certain analyses and documentation is not
available to do those analyses, it is unclear whether the study's
results are accurate. Guidance for statistical surveys requires that a
study's sampling procedures be sufficiently described such that
policymakers can assess whether the study's results can be generalized
to the settings and times of interest.[Footnote 21] The PDC study
documentation mentioned a design that could represent the full
population of PCA-type cases based on sample cases drawn from various
groupings of PCA-type cases.[Footnote 22] However, IRS did not
generalize the study results back to the full PCA case population,
limiting its analysis to the sample cases. It is common for study
results to differ significantly between results for the sampled cases
and results when generalized back to the full population.[Footnote 23]
IRS did not retain sufficient documentation on the sample selection
and analysis to enable it or others at this point to calculate
estimates for the population, measure the margin of error, or
otherwise ensure the soundness of the sample approach.[Footnote 24]
Even though IRS documentation referred to a sampling approach, IRS
officials said the study was not intended to provide generalizable
results. The sample cases assigned to the PCAs and IRS for comparative
purposes may or may not have been similar. IRS was concerned about
potential differences between the groups and tested the difference in
the average balance due amount between the cases to be assigned to
PCAs and IRS. The test did not find a difference. However, this was a
limited test of only one variable. IRS could have tested other
variables, such as taxpayer filing history, adjusted gross income,
filing status, geographic region, and type of forms completed by the
taxpayer. Differences in any of these variables could affect taxpayer
compliance and payment. Performance differences between the PCAs and
IRS could be due to differences in the composition of the groups.
Different Study Objectives and Design Would Have Better Supported the
Program Decision-Making Process:
Because the PDC study had a narrow objective of comparing the results
of collection efforts by IRS and PCAs for the PCA-type cases, its
design did not consider other factors included in federal and other
guidance on conducting program analyses. An objective more directly
focused on providing information to serve as primary support for IRS's
decision on the PDC program could have led IRS to conduct additional
analyses relevant to the decision. The findings from those analyses
could have affected the program decision.
The Study Provides No Statement of the Program's Rationale and
Analysis of Selected Data Related to It:
Guidance on conducting program analysis to determine whether to
continue a program states that the rationale for the government
program being examined should be clearly stated in an analysis.
[Footnote 25] Clearly stating the rationale for the program can help
ensure that the analysis includes relevant factors and outcome
measures, particularly if facts related to the rationale have changed
and therefore need to be included in the analysis. Stating the
rationale of the program and conducting related analyses can help
ensure that variables measured in the study answer the questions
decision makers have about the basic reasons the program exists, the
continuing need for it, and the costs and benefits of continuing,
revising, or ending it. For instance, one rationale for starting the
program was that Congress was unlikely to provide funds for staff to
work these cases. The PDC study did not cover whether this had changed
and, if so, whether IRS would use any portion of additional staffing
for PCA-type cases. As discussed later, the number of staff who would
normally work PCA-type cases has increased in recent years.
Alternatives Were Considered, but Scaling Back the PDC Program Was Not
Among Them:
To inform a decision on whether to continue a program and ensure the
best use of federal resources, government officials need information
on the range of alternatives for achieving a program's objectives at
the lowest costs. Guidance states that program analyses should include
alternative means of achieving program objectives by examining
different program scales, methods of provision, and degrees of
government involvement.[Footnote 26]
The PDC study included three alternatives: (1) PCAs attempting to
collect unpaid tax debts for the inventory assigned to them, (2) IRS
taking collection action on the types of cases that were assigned to
PCAs, and (3) IRS taking collection action on another type of
inventory normally not worked.[Footnote 27] However, beyond not
addressing whether to continue the PDC program, the study did not
analyze alternatives for program scale, such as expanding the PDC
program or scaling it back to a segment of cases that might be more
cost effective for PCAs to work than IRS. For example, in deciding
whether to scale back rather than eliminate the PDC program, IRS could
have analyzed the types of cases, if any, where PDC performance was
better than IRS performance. Assuming that this analysis pointed to
favorable types of cases for PCAs, IRS then could have limited the
program to working those cases, unless the benefit of doing this was
so limited that running a separate PDC program with fixed costs would
not be cost beneficial. According to IRS officials, IRS had
effectively scaled back the program during its course because after
the initial wave of cases assigned to PCAs, IRS had difficulties in
obtaining a sufficient volume of cases appropriate for PCAs to work.
In addition, IRS did not compare the PDC program to what it judged to
be the best overall strategy for improving tax collections, as we had
recommended IRS do in our 2004 report, even though the Commissioner of
Internal Revenue agreed with our recommendation.[Footnote 28] In 2004,
we concluded that IRS should do a study in line with federal guidance,
such as comparing the results of using PCAs to the results from using
the same amount of funds to be paid to PCAs in an unconstrained manner
that IRS determined to be the most effective overall way of achieving
its collection goals. After our report, Congress authorized the PDC
program and included the provision making 25 percent of PDC
collections available to IRS. Conceptually, Congress effectively made
available funds for the PDC program totaling the amounts paid to PCAs
plus the up to 25 percent of PDC collections that IRS could use for
enforcement purposes. We continue to believe that a comparison of
results of the PDC program--as authorized by Congress--to results IRS
would achieve if given the same funds to use in what it judged to be
the best possible manner would have better supported a decision on the
PDC program.
Not All Benefits and Costs Were Included:
To inform a decision on whether to continue a program, government
officials need complete and reliable information on all the program's
benefits and costs.[Footnote 29] Guidance for doing economic analyses
states that to the extent possible all benefits and costs should be
monetized to provide a standard unit of comparison.[Footnote 30] If it
is not feasible to assign monetary values, other quantification of
costs and benefits should be done. If quantification is not possible,
at a minimum, analyses should include a comprehensive listing of the
different types of benefits and costs to identify the full range of
program effects. Furthermore, analyses should be explicit about the
underlying assumptions used to estimate future benefits and costs.
The PDC study measured the government's cost per dollar of direct tax
revenue collected, percentage of balance due collected, and percentage
of cases in payment status at the end of the study period. As cited in
IRS's announcement of the decision to not renew PCAs' contracts, the
study showed that when working the same types of cases as PCAs, IRS
had better results than PCAs. Specifically, IRS's cost was $0.07 to
collect a dollar itself while the government's cost using PCAs was
$0.24 for each dollar collected.[Footnote 31] According to IRS
officials, IRS's results on the cases it worked were comparable to or
better than the results of working its normal cases.
In addition to direct revenues, the study did not estimate or
otherwise discuss indirect revenue (for example, determine whether
other taxpayers are more or less likely to pay their due taxes when
IRS works the cases than when PCAs do). Indirect revenues are
difficult to estimate, but the study could have explored whether any
logical reasons exist to indicate that indirect revenue would have
varied based on which party worked the PCA-type cases.
As discussed further below, the PDC study also did not include
potentially important program costs and benefits for a tax enforcement
program, such as:
* costs to taxpayers to comply with requirements to pay their
delinquent taxes (are costs to taxpayers lower under IRS or PCAs);
* equity (is IRS or are PCAs more effective in collecting taxes from
taxpayers in different income groups, for example); and:
* economic efficiency (is IRS or are PCAs more effective at working
cases involving taxpayers in different industries, for example,
resulting in more or less distortion of activity across types of
businesses).[Footnote 32]
IRS officials said that these benefits and costs were not addressed in
the study because they are difficult to measure. However, IRS could
have followed OMB guidance to, at a minimum, list and discuss these
omitted costs and benefits in the final report. As a result, the
decision to end the program rests, in part, on the assumption that
important differences do not exist between IRS and PCA handling of
these cases related to these costs and benefits. If IRS assumed that
no such differences existed, this should have been stated in the
report with the supporting rationale.
Some important differences might exist. For example, it is not clear
that taxpayer costs would be the same regardless of which party worked
on collecting their tax debt. Taxpayer compliance costs could be lower
when IRS collects taxes because 90 percent of the cases IRS worked
were eligible for systemic actions, such as systemic levies of
taxpayer's assets. Such actions might incur little cost for the
taxpayer beyond reading IRS's notification of intent to levy assets,
as compared to a PCA case, which required that a taxpayer first make a
payment or answer or return a phone call in response to a PCA's letter
or call. Policymakers do not know whether costs to taxpayers differed
between IRS and PCA collections because IRS did not consider this and
other circumstances that could affect taxpayers' costs.
The study also did not include a type of collected revenue--which IRS
called noncommissionable revenue--that IRS officials tracked in a
measure to assess the PDC program's performance and, in some cases,
might have gone uncollected otherwise. For example, IRS did not pay
PCAs a commission when debt was collected within a 10-day window after
being assigned to a PCA. In establishing the program, an IRS official
told us that IRS expected that many dollars would be collected within
this 10-day window because taxpayers would send in payment after
receiving notification that their debts were being assigned to a PCA.
To the extent this occurred, not taking these revenues into account in
the study may have led to an underestimate of the collections
attributable to the PDC program. Other noncommissionable revenue
included collections through actions IRS systemically took regardless
of whether a case was assigned to a PCA, such as refund offsets.
According to IRS officials, such revenues were not included in the PDC
study for either PCAs or IRS collections. IRS data through fiscal year
2007[Footnote 33] show that total collections for the PDC program were
about $32.1 million with $7 million (22 percent) of the total being
noncommissionable; for fiscal year 2008, the total was $37.3 million
with $9.6 million (26 percent) noncommissionable; and for fiscal year
2009, the total was $28.8 million with $11.5 million (40 percent)
noncommissionable.
For the Types of Government Costs IRS Chose to Include, the Study Was
Consistent with Selected OMB Guidance:
IRS's PDC study focused on just a few types of government costs. Those
costs included IRS's costs as well as the commissions paid to PCAs.
Table 2 identifies these types of costs.
Table 2: Summary of the Types of Costs Included in IRS's PDC Study:
IRS costs:
Direct labor (includes number of direct hours worked);
Indirect labor (such as annual and sick leave, training, and
administrative time);
Estimated telephone and mailing/postage costs;
PCA costs:
Commissions on dollars collected[A];
IRS support costs (includes salary and benefits costs for employees)
for:
* referral units and;
* the Taxpayer Advocate Service.
Source: IRS.
[A] Including the commissions as costs was consistent with our
recommendation in GAO-06-1065.
[End of table]
IRS used a variable costing methodology rather than a full costing
approach to compare the costs of PCAs and IRS working inventory that
they determined to be similar.[Footnote 34] According to IRS, the
variable costing approach included expenses that would vary depending
on an increase or decrease in the inventory assigned to either the
PCAs or IRS. The study did not include previous costs, such as costs
in setting up the PDC program. In general, this overall approach to
including costs is consistent with OMB guidance. That guidance says
that analyses should be based on incremental costs and benefits and
sunk costs should be ignored.
IRS did not include management oversight or information technology
since such costs would not vary with the volume of cases handled. For
example, IRS did not include the costs of a PDC program oversight unit
that among other things was responsible for monitoring PCA
performance. Although these costs likely would not vary greatly with
changes in the volume of cases handled by IRS or PCAs, if the absolute
costs were significantly greater either for IRS or PCAs, the
difference may have affected the cost comparison, especially over
relatively small numbers of cases. IRS did not document whether these
costs varied significantly based on who handled the cases.
To determine the cost for the PCAs, IRS used historical cost data of
the PDC program for fiscal year 2008 that were generated from IRS's
Integrated Financial System. IRS apportioned those costs to the number
of cases in its study.
Unlike the PCAs cost data, the IRS costs were not system generated but
were calculated based on estimates and assumptions. For example, IRS
identified the transactions (such as phone calls made or notices sent
to taxpayers) taken on each case in the sample by manually reviewing
individual data posted in the taxpayers' case files in the Integrated
Data Retrieval System. To then calculate direct labor hours per case,
IRS used information from inventory handling time reports and call
handling time reports. IRS determined an average minutes per
transaction, which it applied to the sampled cases in the study.
Because the PCA-type cases were at least somewhat different than cases
ACS normally handled, applying the average minutes from typical
transactions may or may not have accurately reflected the time
actually taken on the PCA-type cases. Other estimated costs were for
telephone calls and postage costs for mailing a single installment
agreement notice to taxpayers.
Study Assessed Past Performance, Not Likely Future Performance:
Another fundamental program analysis principle is to estimate expected
results of the program. Because decisions about the future of a
program depend on the expected future results of it or any
alternatives that are under consideration, future results are to be
included on a discounted basis in either benefit-cost analysis (the
value of expected net benefits, i.e., benefits minus costs) or cost-
effectiveness analysis (to determine which alternative has the lowest
costs for a given amount of benefits or which has the greatest
benefits for a given amount of costs). Past performance can be
relevant in helping to estimate the value of future results.[Footnote
35]
IRS's PDC study included no estimates of future costs and revenues for
the alternatives studied. The study presented data and related ratios,
including costs per dollar collected, only for cases worked in the
past. Although the study text states that the cases studied were like
those PCAs worked during the program and would work in the future, it
provided no analytical basis for this assumption.
Documenting any analyses of future costs and revenues was especially
relevant for any decision on the future of the PDC program because IRS
had previously revised the criteria for selecting PDC cases several
times in order to provide the volume of work to PCAs that had been
anticipated. Accordingly, reasonable questions existed about whether
IRS would be able to continue providing a sufficient stream of work to
PCAs and how that might affect the scope of the program and the nature
of the cases PCAs would work. Absent any such documented analysis,
decision makers and those overseeing the agency had a limited basis on
which to assess whether PCAs likely would have been as effective at
working the cases IRS could deliver in the future as they had been in
working past cases.
IRS officials said that they considered future results, but only
outside the study, in deciding to end the PDC program. IRS officials
said that during management meetings on deciding whether to continue
the PDC program, they considered past program performance compared to
expectations and how that might affect future performance. For
example, they said that they considered gaps between IRS's original
expectations, what the program actually realized, and what the program
would likely achieve in the future given actual program experience.
Among other things, IRS officials noted the declining available case
inventory levels for the program. They said that the program's actual
performance compared to expectations was a reason that IRS ended the
program. IRS was unable to provide documentation of any analyses of
future expected costs and results. Without such analyses, it is
unclear to what extent the program did not meet expectations, whether
IRS determined the underlying reason, and whether the program's future
performance could have been improved in a manner that could have
affected the decision on the program.
IRS Has Not Changed Its Collection Approach Based on Its PDC
Experience and Study, and Whether IRS Will Work Certain PCA-Type Cases
Is Unclear:
Although the Basis Was Not Documented, IRS Concluded That No PCA
Practices Should Be Adopted:
In authorizing the PDC program in 2004, Congress required IRS to
create a measurement plan to capture information on successful
collection techniques used by the contractors that IRS could
adopt.[Footnote 36] The PCAs' best practices were to be compared with
IRS's collection practices.[Footnote 37] IRS was to report on this
measurement plan and its results in a mandated biennial report that
was to include specific types of information.[Footnote 38]
In an unpublished draft biennial report, IRS said it reviewed PCA best
practices and concluded that none of them were sufficiently better
than IRS's practice to merit adoption. IRS officials provided us a
draft version of the biennial report for 2007. IRS neither finalized
the report nor released it. IRS officials pointed to significant
transition in the PDC office during this time, as well as transitions
in the Deputy Commissioner's and the Commissioner's offices. This
draft report described IRS's steps to identify lessons learned from
PCAs.[Footnote 39] It said that IRS did not find any immediate
opportunities to adopt PCA practices, but provided no details beyond
this sentence. The draft report also said IRS would continue to try to
identify lessons learned.
Conflicting Information on Whether or How IRS Will Work PCA-Type Cases:
IRS officials said that IRS had not changed its criteria to start
regularly selecting PCA-type cases to work because the PDC study
results were not sufficient to identify which of the PCA-type cases
could be productively worked. However, IRS officials said that they
were surprised by the study results, which indicated that IRS staff
might have better results working these cases than some of the cases
IRS normally works. IRS officials said that the types of cases sent to
PCAs previously had been considered low priority because of low
potential collection return. In establishing the PDC program, IRS
officials indicated that the PCA contractors would be working inactive
collection cases that IRS collection staff would not be working.
[Footnote 40] For example, in May 2007, after IRS had sent the initial
inventory of cases to PCAs to work, the Acting Commissioner of
Internal Revenue testified that if the money invested in the PDC
program was not used for it, IRS would not use the funds to work cases
that would have been assigned to PCAs.[Footnote 41] Rather, the funds
would be used to work other cases considered higher priority in the
tax debt inventory.
Because of the surprising result, IRS began a pilot study of the types
of cases that had been worked by PCAs. According to IRS officials, the
pilot study's goals were to (1) provide coverage for a segment of the
unpaid tax debt inventory that would not be worked because of the end
of the PCA contract and (2) provide information on which types of
cases IRS could fruitfully work in the future.
IRS placed approximately 19,000 pilot cases with ACS over a 4-month
period beginning in September 2009.[Footnote 42] IRS officials said
that the agency randomly drew the cases from the database of cases
that IRS had used to assign cases to the PCA staff. IRS employees
cannot identify which of their assignments are PCA-type cases. As of
July 2010, the status of these cases was as follows:
* 8,559 cases (45 percent) had been closed, for example, through some
form of payment or other action;
* 4,954 cases (26 percent) were being worked; and:
* 5,552 cases (29 percent) had yet to be worked.
As of June 2010, IRS officials said that they expected that the work
on these pilot study cases would be finished by December 2010. For
these cases, IRS collected data on its activities (such as liens and
levies) and the results (such as dispositions and dollars collected).
Of the 8,559 cases closed as of July 1, 2010, IRS had received payment
in full in 931 cases (11 percent) and had entered into installment
agreements in 2,878 cases (34 percent).
IRS officials said they had planned to use the collected data to
determine if changes should be made to the case selection criteria to
assign certain types of PCA-type cases to the IRS active collection
inventory. According to IRS officials, after the pilot cases were
worked, IRS's Office of Program Evaluation and Risk Analysis (OPERA)
was to study the collected data to see what happened with the cases.
IRS expected that the OPERA study would take a year to complete (i.e.,
until about December 2011). IRS officials said that the agency was
then to determine whether more PCA-type cases should be routinely
worked in IRS and, if so, which ones.
Following guidance on conducting program analyses helps ensure that
relevant information is considered and a sound methodology is used,
including stated study objectives and a clear and appropriate study
design for answering the objectives. As we have reported in the past,
these principles are essential when developing an analysis or
evaluation plan.[Footnote 43] For example, this plan should not only
be written but have sufficient detail to answer questions such as the
following:
* Are the test goals and objectives of IRS's pilot study clearly
stated?
* Has IRS identified the types of data to be collected?
* What specific types of analysis will be performed?
The documentation IRS had available describing the pilot study--a
draft plan prepared in December 2009--addressed summarizing and
providing data on the sampled cases (such as collection action taken,
e.g., liens and levies) and the results (such as the disposition or
current status of the cases and the dollars collected). But as of June
24, 2010, IRS had not produced an approved plan on how it intended to
analyze the pilot study data even though it had worked about three-
fourths of the cases selected for the pilot. According to IRS
officials, progress in developing an analysis plan for the pilot study
had been delayed by efforts to implement Consolidated Decision
Analytics (CDA). Among other things, CDA is to implement new models to
better predict the collection potential of unpaid tax debt cases to
help ensure the best use of IRS's collection resources. Officials said
that any plan for analyzing the pilot study's data would need to
ensure that the pilot study results will be useful given IRS's plans
to more fully implement CDA in January 2011.
Beyond not knowing how the pilot study data will be analyzed, IRS had
not clarified its criteria on how it would use the results of any
analyses to make a decision about assigning more PCA-type cases to be
worked in IRS. Specifically, IRS had no documentation on the criteria
that officials would use in making this decision and what factors, if
any, beyond the analyses of the pilot study data would contribute to
those criteria or that decision. Developing criteria would be
important to ensure that the variables measured in the study would be
useful in supporting the decision on whether to change case selection
criteria to regularly pursue such cases.
As we concluded our review, the status of the pilot study and whether
or how PCA-type cases would be included in active collection case
inventory became less clear. IRS provided conflicting information
about determining whether PCA-type cases have sufficient collection
potential to be included in its collection inventory. On one hand,
officials said that CDA models were built by tracking the
characteristics and collection potential of actual collection cases.
On the other hand, PCA-type cases generally have not been worked by
collection staff, which is why IRS began its pilot study of PCA-type
cases to determine their collection potential. In its comments on our
draft report, IRS said that the CDA models that would be implemented
in January 2011 had overtaken the need to complete the pilot study.
Examining the CDA models was outside the scope of our review.
Therefore, we had not reviewed documentation on what types of data
were used in developing the CDA models. However, in response to our
direct question about whether PCA-type case results were used in
developing the CDA models to be implemented in January 2011, an IRS
official said that they had not been used.
Staff Who Could Work PCA-Type Cases Have Increased, but IRS Has Not
Committed to Working the Cases:
IRS's March 2009 announcement ending the PDC program said that IRS
anticipated hiring more collection staff in fiscal year 2009 and
referred to support from the administration and Congress for increased
IRS enforcement resources. According to IRS officials, IRS's ACS staff
could work on the PCA-type cases. Based on IRS data, ACS staffing
levels--as measured by full-time equivalent (FTE) positions--have
increased since fiscal year 2008.[Footnote 44] The ACS staffing levels
increased by 308 FTEs from fiscal years 2008 to 2009 (i.e., 3,395 to
3,703) and were to increase by 94 FTEs during fiscal year 2010 (i.e.,
to 3,797). These staffing levels account for hiring done to offset
attrition as well as additional hiring beyond attrition.
The PDC program announcement was unclear on whether additional staff
to be hired would be used for PCA-type cases. On the one hand, the
announcement said that "IRS determined the work is best done by IRS
employees," which appears to refer to PCA-type work. On the other
hand, the announcement also said that "new employees would give the
IRS the flexibility to make assignments based on the areas of greatest
need rather than filtering which cases can be worked using contractor
resources." IRS officials told us that the announcement did not imply
that the collection staff to be hired in fiscal year 2009 would be
used to work PCA-type cases.
Conclusions:
In comparing IRS and PCAs working the same types of cases in the PDC
study, IRS was responsive to concerns of some in Congress, the
National Taxpayer Advocate, and others on the study's comparison.
Although a study that adhered to federal guidance on analyzing
programs would have better informed IRS's decision on the fate of the
PDC program, it is not possible to know whether such a study would
have had materially different results or changed IRS's decision on the
program. However it would have provided more complete information for
policymakers to consider. To the extent possible, and especially for
significant program decisions, IRS should adhere to OMB and other
guidance for economic analyses to better ensure that policymakers have
adequate information to support their decisions. IRS does not have
guidance for managers on the types of analyses that should be done and
documented to support program decisions. Such analyses can yield
significant benefits by helping inform decision making, but they also
incur costs. Therefore, careful consideration of the potential risks
and positive impacts of various study designs is necessary to select
an appropriate study design and scope to answer the relevant questions
in a methodologically sufficient manner.
IRS's PDC study suggested that at least certain PCA-type cases, which
IRS had not been working, may be worth including in ACS's inventory
for collection action. To the extent that these results are valid and
reliable, IRS may be able to make a relatively low-cost investment in
certain PCA-type cases to collect tax debts. However, as we concluded
our audit, IRS provided conflicting information about determining
whether PCA-type cases have sufficient collection potential to be
included in its collection inventory. Given the conflicting
information available to us, we believe it is important that PCA-type
case results are considered and incorporated as appropriate into the
CDA models. If IRS determines that completing the pilot study is the
best method to do so, a documented methodology and criteria for the
study's analysis could help IRS make a better decision on which PCA-
type cases, if any, should be added routinely to active collection
status.
Recommendations for Executive Action:
We recommend that the Commissioner of Internal Revenue take the
following three actions:
* Establish guidance on the types of analyses that should be done to
support decisions to initiate, renew, or expand programs. The guidance
might refer to OMB Circular A-94 and, if needed, provide any
supplementation specific to IRS.
* Establish a policy requiring documentation for the design, analyses,
and conclusions of studies supporting program changes.
* Ensure that PCA-type case results are considered and incorporated as
appropriate into the CDA model.
If IRS determines completing the pilot study is the best means to
ensure that PCA-type case results are considered for the CDA models,
the Commissioner should ensure that the pilot study has a documented
methodology and criteria to guide IRS's analysis and decision.
Agency Comments and Our Evaluation:
The IRS Deputy Commissioner for Services and Enforcement provided
written comments on a draft of this report in a September 8, 2010,
letter, which is reprinted in appendix I. IRS staff also provided
technical comments, which we incorporated into the report as
appropriate.
IRS disagreed with our finding that its PDC study was not soundly
designed to support its decision on whether to continue contracting
out debt collection. IRS said the study's comparison of the cost-
effectiveness of PCAs and IRS working similar cases provided
meaningful data that aided its decision making. IRS cited an
independent review of the PDC study that found the results to be
reasonable, even though the study had limitations and constraints. We
continue to believe that the study was not a soundly designed cost-
effectiveness comparison for supporting IRS's decision. Our report
discusses our reasoning in detail, focusing on the study's
methodological errors, narrow scope, and lack of adherence to guidance
for doing such studies. For example, IRS did not do the analysis
necessary to generalize the study results to the full PCA case
population even though study results could differ significantly when
generalized to the full population. Our meetings with staff who
performed the independent review and our analyses of their
documentation did not change our finding about IRS's study.
IRS agreed with our two recommendations dealing with establishing
guidance on analyses to support decisions to initiate, renew, or
expand a program and policies to ensure documentation of such studies.
More specifically, IRS said it would review current guidance and
policies and develop additional guidance where needed.
IRS agreed in principle with our third draft recommendation on
ensuring that a documented methodology and criteria guide IRS's
analysis and decision on whether to include selected PCA-type cases in
its collection inventory, but said events have overtaken the need to
complete the ongoing study, citing IRS's plans to implement CDA models
in fiscal year 2011. These models are intended to select cases with
the best potential for collection action in one of IRS's work streams.
IRS said that to measure the impact of the PCA-type cases, as was the
plan when the PCA project was terminated, is no longer necessary.
We had discussed with IRS officials the continued need for the pilot
study when IRS told us in July 2010 that it planned to implement CDA
in January 2011. IRS officials, including the Acting Director,
Collection Business Reengineering, said that while CDA selection would
focus on collection potential and not type of case (i.e., PCA-type),
the pilot study of approximately 19,000 PCA-type cases might provide
data useful for improving CDA models. Officials affirmed that they
initiated the pilot study because the PDC study showed that PCA-type
cases might have high collection potential at low cost. Accordingly,
our draft report recommended that IRS document the methodology and
criteria for its pilot study. Information provided in IRS comments on
the report and in response to our subsequent questions suggests that
whether and how PCA-type cases may be selected for active collection
inventory is uncertain. Although IRS's comments on the draft report
said that the need for completing the pilot case study was overtaken
by the development of the CDA models, in separate technical comments
IRS officials said they were continuing to work the pilot cases and
provided no indication that they would stop working them before CDA is
implemented in January 2011. Further, in response to our question
about whether PCA-type case results were used in developing the CDA
models, an IRS official said that they had not been used. In response
to IRS's comments and absent evidence that CDA will be implemented as
planned and that its models will include IRS's experience in
attempting collection of PCA-type cases, we revised the third
recommendation to better focus on ensuring that PCA-type case results
are considered and incorporated as appropriate into the CDA models.
Further, if IRS determines completing the pilot study is the best
means to ensure that PCA-type case results are considered for the CDA
models, we maintained our recommendation that IRS ensure that the
study has a documented methodology and criteria to guide IRS's
analysis and decision.
As agreed with your office, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 30 days
from the report date. At that time, we will send copies to the
Secretary of the Treasury, the Commissioner of Internal Revenue, and
other interested parties. In addition, the report will be available at
no charge on the GAO Web site at [hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-9110 or brostekm@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. Key contributors to this report are
listed in appendix II.
Sincerely yours,
Signed by:
Michael Brostek:
Director, Tax Issues:
Strategic Issues:
[End of section]
Appendix I: Comments from the Internal Revenue Service:
Department Of The Treasury:
Deputy Commissioner
Internal Revenue Service:
Washington, D.C. 20224:
September 8, 2010:
Mr. Michael Brostek:
Director, Tax Issues:
U.S. Government Accountability Office:
Washington, D.C. 20548:
Dear Mr. Brostek:
Thank you for the opportunity to review the Government Accountability
Office's (GAO) draft report entitled, "Tax Debt Collection: IRS Could
Improve Ongoing and Future Studies by Establishing Appropriate
Guidance" (GA0-10-963).
As the report acknowledges, the IRS's study of Private Debt Collection
(PDC) program operations was designed to compare the cost of working
similar cases in IRS Automated Collection System (ACS) call centers
with Private Collection Agencies (PCAs). While the study's objectives
were limited to comparing the cost-effectiveness of PCAs and ACS, the
study provided meaningful data to help make the overall decision
on whether to continue the PDC program. Data from the study along with
other resource, inventory, and operational factors were considered in
the overall decision.
The IRS disagrees with the report's contention that the study was not
soundly designed. An independent review of the cost-effectiveness
study confirmed that while the study contained limitations and
constraints, the findings presented in the study were reasonable. The
independent review found that it was unlikely that any of the
constraints identified would reverse the comparative positions of ACS
and PDC operations.
The report provides recommendations for improving the guidance
available to managers on how to perform program analysis. We agree
that clearer guidance would improve consistency, documentation, and
the overall quality of future program reviews.
The enclosed response addresses each recommendation separately.
If you have questions, please contact me, or a member of your staff
may contact Nikole Flax, Assistant Deputy Commissioner for Services
and Enforcement, at (202) 622-6860.
Sincerely,
Signed by:
Steven T. Miller:
Enclosure:
[End of letter]
Enclosure:
GAO Recommendations and IRS Response to GAO Draft Report:
Tax Debt Collection: IRS Could Improve Ongoing and Future Studies by
Establishing Appropriate Guidance: GA0-10-963:
Recommendation 1:
Establish guidance on the types of analyses that should be done to
support decisions to initiate, renew, or expand programs. The guidance
might refer to OMB Circular A-94 and, if needed, provide any
supplementation specific to IRS.
Comments:
We agree with the recommendation that the IRS explore clearer guidance
on the types of analyses that should be done to support decisions that
will have a significant impact on major programs. IRS's Office of
Program Evaluation and Risk Analysis (OPERA) will lead a review of
current IRS policies for performing program analysis and develop
additional guidance where needed.
Recommendation 2:
Establish a policy requiring documentation for the design, analyses,
and conclusions of studies supporting program changes.
Comments:
We agree with the recommendation that the IRS explore clearer guidance
on the documentation requirements for analysis conducted to support
significant program changes. IRS's Office of Program Evaluation and
Risk Analysis (OPERA) will lead a review of current IRS policies for
documenting program analysis and develop additional guidance where
needed.
Recommendation 3:
Ensure that IRS's ongoing study on whether to include selected PCA-
type cases in the collection inventory includes a documented
methodology and criteria to guide IRS's analysis and decision.
Comments:
We agree with this recommendation in principle, but events have
overtaken the need to complete this study. In FY 2011, the IRS will
implement Consolidated Decision Analytics (CDA) to select balance due
cases to be worked and route the cases to the appropriate work stream.
The CDA system will use decision analytics and modeling to select
cases with the best potential for collection and will put these cases,
in priority order, into the work streams of ACS or field collection,
as indicated by the models. The models and their output will be
regularly reviewed for accuracy and performance. As such, we believe
that using these models will force the best cases into our work
streams, and to measure the impact of the PCA-type cases, as was the
plan when the PCA project was terminated, is no longer necessary.
[End of section]
Appendix II: GAO Contact and Staff Acknowledgments:
GAO Contact:
Michael Brostek, (202) 512-9110 or brostekm@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Tom Short, Assistant Director;
Ray Bush; George Guttman; Lois Hanshaw; Ronald W. Jones; Veronica
Mayhand; Ed Nannenhorn; Karen O'Conor; and Cynthia Saunders made key
contributions to this report.
[End of section]
Footnotes:
[1] 26 U.S.C. § 6306(c)(1).
[2] The National Taxpayer Advocate is the head of the Taxpayer
Advocate Service within IRS, which works to assist taxpayers in
resolving individual problems and propose administrative and
legislative changes to mitigate problems affecting groups of taxpayers.
[3] 26 U.S.C. § 6306 note (American Jobs Creation Act of 2004, Pub. L.
No. 108-357, title VIII, § 881(e), 118 Stat. 1418 (Oct. 22, 2004)).
[4] When this report refers to working cases or inventory, we are
using IRS's term for any case action taken to collect unpaid tax debt.
[5] According to the contractor's staff, they had approximately 3
weeks to validate the study's conclusion that IRS was more cost
effective in collecting this debt than PCAs. The study's conclusion
was limited to the measures and data IRS chose to study.
[6] We requested comments about IRS's study from the two PCAs with IRS
contracts at the time IRS released the study, of which one provided
comments to us.
[7] OMB Circular A-94, Guidelines and Discount Rates for Benefit-Cost
Analysis of Federal Programs (Washington, D.C., Oct. 29, 1992). The
goal of Circular A-94 is to promote efficient federal resource
allocation through well-informed decision making. It applies to
analyses used to support federal decisions to initiate, renew, or
expand programs or projects resulting in a series of measurable
benefits or costs extending 3 or more years into the future. The
circular is to serve as a checklist for whether an agency has
considered and properly dealt with all the elements for sound benefit-
cost and cost-effectiveness analyses. The circular's guidelines are
suggested for use in the internal planning of executive branch
agencies.
[8] GAO, The Evaluation Synthesis, [hyperlink,
http://www.gao.gov/products/GAO/PEMD-10.1.2] (Washington, D.C.: March
1992).
[9] Joel Slemrod and Shlomo Yitzaki, The Costs of Taxation and the
Marginal Efficiency Cost of Funds, International Monetary Fund Staff
Papers (March 1996).
[10] [hyperlink, http://www.gao.gov/products/GAO/PEMD-10.1.2].
[11] GAO, Earned Income Tax Credit: Implementation of Three New Tests
Proceeded Smoothly, But Tests and Evaluation Plans Were Not Fully
Documented, [hyperlink, http://www.gao.gov/products/GAO-05-92]
(Washington, D.C.: Dec. 30, 2004).
[12] GAO, Tax Debt Collection: IRS Is Addressing Critical Success
Factors for Contracting Out but Will Need to Study the Best Use of
Resources, [hyperlink, http://www.gao.gov/products/GAO-04-492]
(Washington, D.C.: May 24, 2004).
[13] [hyperlink, http://www.gao.gov/products/GAO-04-492].
[14] GAO, Tax Debt Collection: IRS Needs to Complete Steps to Help
Ensure Contracting Out Achieves Desired Results and Best Use of
Federal Resources, [hyperlink,
http://www.gao.gov/products/GAO-06-1065] (Washington, D.C.: Sept. 29,
2006).
[15] Internal Revenue Service, IRS Private Debt Collection Cost
Effectiveness Study (Washington, D.C., March 2009).
[16] ACS handles IRS collections by telephone whereas its Collection
Field Function handles more complex and in-person collection contacts.
The study also included taxpayer satisfaction scores, but the study
report states that the scores reflect cases beyond those in the study
and should not be used to compare the performance of PCAs and IRS.
[17] See [hyperlink, http://www.congress.gov/cgi-
lis/query/z?c111:H.R.796]: for H.R. 796, 111th Cong. (2009).
[18] Omnibus Appropriations Act, 2009, Pub. L. No. 111-8, 123 Stat.
524 (Mar. 11, 2009).
[19] 26 U.S.C. § 7801 note (Treasury, Postal Service and General
Government Appropriations Act, 1995, Pub. L. No. 103-329, 108 Stat.
2382 (Sept. 30, 1994), as amended).
[20] Consolidated Appropriations Act, 2010, Pub. L. No. 111-117, 123
Stat. 3034 (Dec. 16, 2009).
[21] See White House, Standards and Guidance for Statistical Surveys,
[hyperlink, http://www.whitehouse.gov/omb/inforeg_statpolicy/#pr]
(Washington, D.C., September 2006) (accessed Aug. 3, 2010). According
to Standard 7.3, agencies must produce survey documentation that
includes those materials necessary to understand how to properly
analyze data from each survey, as well as the information necessary to
replicate and evaluate each survey's results (see also Standard 1.2).
Survey documentation must be readily accessible to users, unless it is
necessary to restrict access to protect confidentiality.
[22] The documentation that is available suggests that IRS drew a
stratified random sample. Such a sample design involves first
classifying the population into several groups--strata--and then
taking a random sample from each stratum.
[23] In a stratified sample, sample cases in different strata can
represent a substantially different number of cases. For example, if
the cases in the population are divided into two strata based on
characteristics that are of interest, a case in one stratum might
represent 20 cases in its stratum whereas a case in the other stratum
might represent 100 cases in its stratum. In order to generalize such
a stratified sample to the full population, cases must be "weighted"
based on the numbers of cases in the population they represent.
Largely for this reason, analytic results--as in the PDC study--that
are based only on the sampled cases themselves may be significantly
different than they would be when properly generalized back to the
full universe from which the sample was drawn.
[24] Sampling errors are errors associated with survey estimates that
are due to sampling some and not all of the units in the sampling
frame.
[25] OMB Circular A-94.
[26] OMB Circular A-94.
[27] The PDC study called the third alternative the "next best use of
funds." However, IRS officials said that they did not decide to study
these cases because they believed working such cases would be the most
effective and efficient strategy for achieving overall collection
goals. Instead, IRS officials restricted the types of strategies that
could be selected. For example, IRS officials considered attempting
collection only on cases suitable for IRS's ACS, which handles IRS
collections by telephone and not other collection strategies like
information systems investments, or assigning more cases to IRS's
Collection Field Function (which handles more complex and in-person
collection contacts). In deciding which types of ACS cases to study,
IRS officials also gave higher priority to cases that--like the PCA-
type inventory--otherwise would not have been worked. For example, IRS
officials considered the alternative of including more of the normal
ACS inventory but instead chose to study cases not normally in ACS's
active inventory, in part because the officials wanted to learn more
about the potential costs and collection results for them. IRS
normally does not take collection action on such cases because they
require more manual processing than is deemed worthwhile. Regardless,
IRS did not cite the results of these cases in its decision to end the
PDC contracts or ask the independent reviewers of the PDC study to
address the validity of the study's results for these cases.
[28] [hyperlink, http://www.gao.gov/products/GAO-04-492].
[29] From an economic perspective, tax revenues are not a benefit to
society in that they represent simply a transfer of funds from
taxpayers to the government. Social benefits and social costs, not
just benefits and costs to the government, are the relevant benefits
and costs.
[30] OMB Circular A-94.
[31] For the third alternative IRS studied, government costs to
collect a dollar were $0.17 and $0.25, depending on which IRS division
worked the cases (Small Business/Self-Employed Division and Wage and
Investment Division, respectively).
[32] Noncompliance can lead to economic distortions as more resources,
such as workers, are attracted to areas of the economy where
noncompliance is relatively high. If PCAs were more effective than IRS
in collecting taxes in particular industries or from particular types
of business that are highly noncompliant, it might be possible to
justify keeping PCAs on economic efficiency grounds, even if IRS was
more effective across all the studied cases.
[33] Beginning September 7, 2006, when IRS turned over the first cases
to PCAs.
[34] Variable costs fluctuate in proportion with changes in production.
[35] OMB guidance states that retrospective studies are potentially
valuable in determining the necessary corrections in existing programs
and improving future estimates of results.
[36] 26 U.S.C. § 6306 note (American Jobs Creation Act of 2004, Pub.
L. No. 108-357, title VIII, § 881(e), 118 Stat. 1418 (Oct. 22, 2004)).
[37] We previously reported that part of Congress's intent in
approving the 1997 IRS PDC pilot program was to learn more about
private sector collection techniques. Additionally, we stated that the
program's measurement plan did not include a comparison of such best
practices and the written design lacked a mechanism to capture
information on the best collection practices used by PCAs that could
be adopted by IRS. See GAO, Internal Revenue Service: Issues Affecting
IRS' Private Debt Collection Pilot, [hyperlink,
http://www.gao.gov/products/GAO/GGD-97-129R] (Washington, D.C.: July
18, 1997).
[38] In addition to the information on PCA best practices, Congress
required that the biennial report include a cost-benefit analysis, the
impacts of PCA contracts on IRS collection enforcement staffing and on
unpaid assessments and assessments collected by IRS after initial PCA
contacts, the amounts collected and collection costs incurred by IRS,
an evaluation of contractor performance, and a disclosure safeguard
report.
[39] Among other things, the draft biennial report for 2007 also
included an evaluation of contractor performance.
[40] Inactive cases were cases for which IRS deferred or suspended
collection action because of (1) IRS resource limitations, (2) the
cases not meeting IRS tolerance levels for active inventory, or (3)
IRS not being able to locate or contact the taxpayers.
[41] Statement of Acting Commissioner of Internal Revenue Kevin M.
Brown before the House Ways and Means Committee on May 23, 2007.
[42] The figures IRS provided for the total cases ranged from 18,134
to 19,065. IRS provided no explanation for the discrepancies in
response to our requests.
[43] [hyperlink, http://www.gao.gov/products/GAO-05-92].
[44] According to IRS, a FTE is the equivalent of one person working
full-time for 1 year without overtime.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
