Tax Administration
IRS Needs to Strengthen Its Approach for Evaluating the SRFMI Data-Sharing Pilot Program Gao ID: GAO-09-45 November 7, 2008The State Reverse File Match Initiative (SRFMI) is one of the Internal Revenue Service's (IRS) data-sharing strategies to reduce the estimated $345 billion gross federal tax gap. SRFMI matches federal and state taxpayer data to identify noncompliant taxpayers--individuals and businesses who do not file tax returns or do not report all of their income. IRS's document-matching program has proven to be a highly cost-effective way of identifying underreported income, thereby bringing in billions of dollars of tax revenue while boosting voluntary compliance. Based on concerns about whether IRS is fully using data from state and local governments to reduce the tax gap, GAO was asked to assess IRS's plan for evaluating the effectiveness of the SRFMI pilot taxpayer data-sharing program. To address these objectives, GAO consulted social science and evaluation literature, published GAO guidance, and IRS guidance; reviewed and analyzed SRFMI planning and evaluation documents; and interviewed IRS officials about IRS's plans to evaluate SRFMI.
IRS is developing a plan for evaluating SRFMI data but does not currently have a documented plan even as the agency enters phase III of the pilot program and is less than 1 year away from rolling out the pilot to 45 states. According to IRS officials, the SRFMI pilot includes plans to evaluate program results to make decisions about expanding data sharing with states and using compliance data to assess whether SRFMI cases are more or less productive than other cases. IRS has little documentation on its evaluation. A well-developed and documented plan can help ensure that the evaluation generates performance information needed to make effective management decisions. A sound evaluation approach should also include criteria or standards for determining pilot-program performance. However, IRS has no criteria or standards for determining where the pilot program performs adequately to be incorporated into normal IRS compliance processes. IRS officials told GAO that they plan to use research results to help formulate standards to determine pilot success but did not provide a target date when this will occur. Moreover, IRS has not completely articulated its methodology to evaluate the pilot program. IRS officials have action plans and testing sample plans for individual units. The action plans primarily contained project-management-type information such as actions or tasks to be taken by individuals, start date, completion date, and comments rather than a comprehensive description of the methodological approach for the overall pilot project. The testing sample plans were questionnaires soliciting information about compliance measures for the individual unit and the resources available for testing SRFMI data. None of the plans outline the methods, timing, or frequency of data collection. While IRS officials have begun formulating plans for developing a sampling approach and determining appropriate sample size, they encountered challenges such as delays in information-technology assistance and time limits for using taxpayer data that have impeded progress in moving forward on its evaluation methodology. The need to evaluate the program is underscored because obtaining and using SRFMI data imposes costs not only on IRS but also on the states. Without a sound assessment of pilot program results, IRS may make poor decisions about the program's future.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-09-45, Tax Administration: IRS Needs to Strengthen Its Approach for Evaluating the SRFMI Data-Sharing Pilot Program
This is the accessible text file for GAO report number GAO-09-45
entitled 'Tax Administration: IRS Needs to Strengthen Its Approach for
Evaluating the SRFMI Data-Sharing Pilot Program' which was released on
December 6, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Committee on Finance, U.S. Senate:
United States Government Accountability Office:
GAO:
November 2008:
Tax Administration:
IRS Needs to Strengthen Its Approach for Evaluating the SRFMI Data-
Sharing Pilot Program:
Tax Administration:
GAO-09-45:
GAO Highlights:
Highlights of GAO-09-45, a report to the Committee on Finance, U.S.
Senate.
Why GAO Did This Study:
The State Reverse File Match Initiative (SRFMI) is one of the Internal
Revenue Service‘s (IRS) data-sharing strategies to reduce the estimated
$345 billion gross federal tax gap. SRFMI matches federal and state
taxpayer data to identify noncompliant taxpayers”individuals and
businesses who do not file tax returns or do not report all of their
income. IRS‘s document-matching program has proven to be a highly cost-
effective way of identifying underreported income, thereby bringing in
billions of dollars of tax revenue while boosting voluntary compliance.
Based on concerns about whether IRS is fully using data from state and
local governments to reduce the tax gap, GAO was asked to assess IRS‘s
plan for evaluating the effectiveness of the SRFMI pilot taxpayer data-
sharing program. To address these objectives, GAO consulted social
science and evaluation literature, published GAO guidance, and IRS
guidance; reviewed and analyzed SRFMI planning and evaluation
documents; and interviewed IRS officials about IRS‘s plans to evaluate
SRFMI.
What GAO Found:
IRS is developing a plan for evaluating SRFMI data but does not
currently have a documented plan even as the agency enters phase III of
the pilot program and is less than 1 year away from rolling out the
pilot to 45 states. According to IRS officials, the SRFMI pilot
includes plans to evaluate program results to make decisions about
expanding data sharing with states and using compliance data to assess
whether SRFMI cases are more or less productive than other cases. IRS
has little documentation on its evaluation. A well-developed and
documented plan can help ensure that the evaluation generates
performance information needed to make effective management decisions.
A sound evaluation approach should also include criteria or standards
for determining pilot-program performance. However, IRS has no criteria
or standards for determining where the pilot program performs
adequately to be incorporated into normal IRS compliance processes. IRS
officials told GAO that they plan to use research results to help
formulate standards to determine pilot success but did not provide a
target date when this will occur.
Moreover, IRS has not completely articulated its methodology to
evaluate the pilot program. IRS officials have action plans and testing
sample plans for individual units. The action plans primarily contained
project-management-type information such as actions or tasks to be
taken by individuals, start date, completion date, and comments rather
than a comprehensive description of the methodological approach for the
overall pilot project. The testing sample plans were questionnaires
soliciting information about compliance measures for the individual
unit and the resources available for testing SRFMI data. None of the
plans outline the methods, timing, or frequency of data collection.
While IRS officials have begun formulating plans for developing a
sampling approach and determining appropriate sample size, they
encountered challenges such as delays in information-technology
assistance and time limits for using taxpayer data that have impeded
progress in moving forward on its evaluation methodology. The need to
evaluate the program is underscored because obtaining and using SRFMI
data imposes costs not only on IRS but also on the states. Without a
sound assessment of pilot program results, IRS may make poor decisions
about the program‘s future.
What GAO Recommends:
GAO recommends that the Commissioner of Internal Revenue ensure that
IRS develops an evaluation plan to accurately and reliably assess the
SRFMI pilot program‘s results. This plan should address all components
of the program and include key evaluation features. IRS agreed with our
recommendation.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-09-45]. For more
information, contact Michael Brostek, (202) 512-9110.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Although IRS Has Begun Developing One, IRS Does Not Yet Have a Sound
Evaluation Plan for SRFMI:
Conclusions:
Recommendation for Executive Action:
Agency Comments:
Appendix I: States Participating in Phases I, II, and III:
Appendix II: Overview of Data-Matching Process and Further Examinations
or Investigations under SRFMI:
Appendix III: Preliminary Compliance Measures by ASFR and Field
Examination Activities:
Appendix IV: Comments from the Internal Revenue Service:
Appendix V: GAO Contact and Staff Acknowledgments:
Table:
Table 1: List of States Participating in State Reverse File Match
Initiative (SRFMI) Pilot Program:
Figures:
Figure 1: Overview of Data Matching under SRFMI Pilot Program:
Figure 2: Overview of the Compliance Activities for Further Examination
or Investigation:
Abbreviations:
ASFR: Automated Substitute for Return:
ATAT: Abusive Tax Avoidance Transactions:
GAO: Government Accountability Office:
GLDEP: Governmental Liaison Data Exchange Program:
IRS: Internal Revenue Service:
QETP: Questionable Employment Tax Practice:
SB/SE: Small Business/Self Employed:
SB/SE Research: Small Business/Self Employed Research:
SRFMI: State Reverse File Match Initiative:
TIN: Taxpayer Identification Number:
USCIS: United States Citizenship and Immigration:
Services:
United States Government Accountability Office:
Washington, DC 20548:
November 7, 2008:
The Honorable Max Baucus:
Chairman:
The Honorable Charles E. Grassley:
Ranking Member:
Committee on Finance:
United States Senate:
The State Reverse File Match Initiative (SRFMI) is one of the Internal
Revenue Service's (IRS) data-sharing strategies to reduce the estimated
$345 billion gross federal tax gap. SRFMI matches federal and state
taxpayer data to identify noncompliant federal taxpayers--individuals
and businesses who do not file tax returns or do not report all of
their income. Data sharing allows agencies to leverage external
partnerships with other agencies and has proven to be a valuable tool
to a number of federal agencies striving to improve decisions about
applicants' eligibility for federal programs. As we have previously
found, federal agencies are increasingly sharing taxpayer data to help
verify applicant-provided information.[Footnote 1] IRS's document-
matching program, for example, has proven to be a highly cost-effective
way of identifying underreported income, thereby bringing in billions
of dollars of tax revenue while boosting voluntary compliance.[Footnote
2]
In 2007, you expressed concerns about whether IRS is fully using data
from state and local governments to reduce the tax gap. Related to this
interest, you requested that we assess IRS's plan for evaluating the
effectiveness of the SRFMI pilot taxpayer data-sharing program. To
respond to your request, we (1) consulted social science and evaluation
literature, along with published GAO guidance and IRS guidance, to
identify key features of an evaluation plan; (2) reviewed and analyzed
SRFMI planning and evaluation documents to determine whether they
contained key features of a sound evaluation plan; and (3) interviewed
relevant IRS officials about IRS's plans to evaluate SRFMI, including
what cost data IRS plans to use to help fully understand how productive
SRFMI cases are.
We conducted this performance audit from July through September 2008,
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objective. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objective.
Results in Brief:
IRS is developing a plan for evaluating SRFMI data but does not
currently have a documented plan even as the agency enters phase III of
the pilot program and is less than 1 year away from rolling out the
pilot to 45 states. According to IRS officials, the SRFMI pilot
includes plans to evaluate program results to make decisions about
expanding data sharing with states and using compliance data to assess
whether SRFMI cases are more or less productive than other cases. IRS
has little documentation on its evaluation approach. A well-developed
and documented plan can help ensure that the evaluation generates
performance information needed to make effective management decisions.
A sound evaluation should also include criteria or standards for
determining pilot-program performance. However, IRS has no criteria or
standards for determining where the pilot program performs adequately
to be incorporated into normal IRS compliance processes. IRS officials
told us that they plan to use research results to help formulate
standards to determine pilot success but did not provide a target date
when this will occur.
Moreover, IRS has not completely articulated its methodology to
evaluate the pilot program. IRS officials have action plans and testing
sample plans for individual units. The action plans primarily contained
project-management-type information such as actions or tasks to be
taken by individuals, start date, completion date, and comments, rather
than a comprehensive description of the methodological approach for the
overall pilot project. The testing sample plans were questionnaires
soliciting information about compliance measures for the individual
unit and the resources available for testing SRFMI data. None of the
plans outline the methods, timing, or frequency of data collection.
While IRS officials have begun formulating plans for developing a
sampling approach and determining appropriate sample size, IRS
officials encountered challenges such as delays in information-
technology assistance and time limits for using taxpayer data that have
impeded progress in moving forward on developing its evaluation
methodology. The need to evaluate the program is underscored because
obtaining and using SRFMI data imposes costs not only on IRS but also
on the states. Without a sound assessment of pilot-program results, IRS
may make poor decisions about the program's future.
We recommend that the Commissioner of Internal Revenue ensure that IRS
develops an evaluation plan to accurately and reliably assess the SRFMI
pilot program's results. This plan should address all components of the
pilot program and include at a minimum key evaluation features of a
sound plan:
* well-defined, clear, and measurable objectives;
* criteria or standards for determining pilot-program performance;
* clearly articulated methodology, including sound sampling methods,
determination of appropriate sample size for the evaluation design, and
a strategy for comparing the pilot results with other efforts;
* a clear plan that details the type and source of data necessary to
evaluate the pilot, methods for data collection, and the timing and
frequency of data collection; and:
* a detailed data-analysis plan to track the program's performance and
evaluate the final results of the project.[Footnote 3]
In commenting on a draft of this report (see app. IV) on behalf of the
Commissioner of Internal Revenue, the Deputy Commissioner for Services
and Enforcement agreed with our recommendation to develop an evaluation
plan to accurately and reliably assess the SRFMI pilot program's
results. The Deputy Commissioner said that a summit will be conducted
with the business unit owners and their respective research functions
to further develop and enhance their existing SRFMI plans to include
the key evaluation features we recommend. In addition, IRS agreed to
develop an overall evaluation plan to accurately and reliably assess
all components of the SRFMI pilot program's results and include the key
evaluation features cited above.
Background:
IRS and States Have an Existing Data-Sharing Relationship:
Data-sharing programs between IRS and states have been in place for
many years. Historically, the flow of information was largely from the
federal government to the states. IRS shares tax data with the state
revenue agencies based on the states' needs, and the states use the
data for tax-administration purposes. In recent years, however, IRS has
taken steps to change the flow of information and pursue additional
opportunities to obtain and use states' taxpayer data for tax-
compliance purposes. Some examples of reciprocal data sharing between
IRS and state agencies include:
* Governmental Liaison Data Exchange Program (GLDEP)--IRS and state
revenue agencies exchange taxpayer data for tax
administration.[Footnote 4]
* Questionable Employment Tax Program (QETP)--IRS shares Form 1099-
MISC[Footnote 5] extract data with the state workforce agencies, and
the state workforce agencies investigate whether the people on IRS's
list are employees or independent contractors.
* Abusive Tax Avoidance Transactions (ATAT)--IRS and states exchange
information on adjustments made during tax shelter audits. According to
IRS, this exchange has led to the discovery by IRS and states of tens
of millions of dollars in previously unknown fraudulent tax avoidance
schemes.
SRFMI Pilot Program Matches State Data with IRS Taxpayer Data to
Identify Nonfilers and Underreporters:
In calendar year 2006, IRS saw an opportunity to expand data sharing by
obtaining more information from states. Accordingly, it created the
SRFMI pilot program. SRFMI has a goal to reduce the federal tax gap by
improving the tax compliance of individuals and businesses who do not
file (nonfilers) or who do not report all of their income
(underreporters). Under SRFMI, states match IRS's return-filing
information received through the existing GLDEP data-sharing
arrangement against state tax data to identify individuals and
businesses who filed a state return but did not file a federal return
or reported income to the state but not to IRS.[Footnote 6] The pilot
uses four different types of tax-return data--individual, corporate,
sales, and withholding--and takes steps to validate or put state data
into a format useable for compliance activities.[Footnote 7] (App. II,
fig. 1 has an overview of data matching under the SRFMI program.)
IRS Uses the Data-Matching Results in Several Different Compliance
Activities:
The result of matching federal and state tax data is a file with data
on potential federal nonfilers and underreporters. IRS then filters the
matched data to determine which matching results represent potential
noncompliance or "leads" for compliance activities in several different
organization units. These include:
* Automated Substitute For Return (ASFR)--creates a substitute federal
tax return where none exists and makes an assessment.
* Automated Underreporter--compares filed returns with information-
reporting returns to identify unreported income. For the SRFMI pilot,
filed federal returns will be compared to filed state returns and not
necessarily to information returns.
* Field Examination--conducts face to face examinations.
* Campus Examination--conducts correspondence audits generally
involving basic single issues.
* Specialty Tax--conducts examinations on excise taxes, employment
taxes, and estate and gift taxes.
* Collection--collects revenue and secures delinquent returns.
* Fraud/Bank Secrecy Act--assists other functions in developing civil
and criminal fraud investigations.
* Criminal Investigation--conducts investigations on criminal
violations of the Internal Revenue Code and related financial crimes.
Each compliance activity determines which of the leads will be
designated for examination, contacted by correspondence requesting
additional information, or referred for further investigation for
criminal activity. (App. II, fig. 2 has an overview of the distribution
of SRFMI data for further examinations.)
The Pilot Program Has Four Phases with Increasing Numbers of
Participating States:
Three states participated in phase I, 6 additional states participated
in phase II, and 14 agencies from 13 states participated in phase III.
(For a table showing the states participating in phases I, II, and III,
see app. I.) IRS has enrolled 45 states to deliver SRFMI files in July
2009 and has the following schedule for data exchanges:
* Phase I started in March 2006.
* States provided SRFMI extracts for phase II in July 2007.[Footnote 8]
* States provided SRFMI extracts for phase III in July 2008.
* States are to provide extracts for nationwide rollout by July 15,
2009.
Although IRS Has Begun Developing One, IRS Does Not Yet Have a Sound
Evaluation Plan for SRFMI:
According to IRS officials, the SRFMI pilot includes plans to evaluate
program results to make decisions about expanding data sharing with
states and using compliance data to evaluate whether SRFMI cases are
more or less productive than other cases. IRS's approach to
implementing the pilot is outlined in a series of briefing documents.
IRS officials told us that since the program began in 2006, IRS has:
* worked with states to receive state data in a uniform record layout;
* validated and perfected federal and state data matches for the nine
states in phases I and II;[Footnote 9]
* distributed data matches to several business units;
* conducted compliance activities and tabulated phase I activities for
two organization units--ASFR and field examination; and:
* commissioned the Small Business/Self Employed Research (SB/SE
Research) unit to provide technical assistance in the areas of data
analysis, methodology, and project design; to provide various
tabulations of matched data and analyze the usefulness of the matches
for identifying nonfilers and underreporters; to develop appropriate
sample sizes for each compliance activity; and to conduct studies.
Using phase I data, SB/SE Research compared preliminary results of
SRFMI data used by two units in the SB/SE Division. ASFR reported on
closed cases, taxpayer returns, defaults, reconsiderations, dollars
assessed, and dollars assessed per return. Field examination reported
on more data elements than ASFR, including no-change rate, total
nonexamined cases, and average hours spent on returns. (App. III shows
preliminary compliance measures used by ASFR and field examination
activities.) The other compliance units have not reported results
because they have not begun to work with the matched data.
IRS officials told us that early results are limited and not
statistically sound but said that these results helped them become
familiar with how the data should be filtered and with compliance
results information. SB/SE Research has provided ongoing support and
consultation and has conducted some research studies to help develop an
approach for evaluating SRFMI pilot results. For example, SB/SE
Research matched Colorado and New Jersey tax-amnesty data to compare
the potential use of resources for SRFMI cases with non-SRFMI cases and
found that state tax-amnesty files cannot be used to definitively
identify federal nonfilers or underreporters and that the data lacked
fields needed to determine filing requirements and self-employment
status.[Footnote 10] However, it found that state tax-amnesty data can
be a source for potential high-income nonfiler leads and case building,
with limitations. The Colorado and New Jersey data were not SRFMI data
but were the only data available for a first test of the SRFMI concept.
A study of phase I Arkansas, Iowa, and Massachusetts SRFMI ASFR and
Field Examination cases found mixed results. SRFMI ASFR cases most
clearly were less productive than regular cases, because both the
dollars assessed per case and per hour were substantially lower for
SRFMI than non-SRFMI cases. Field Examination SRFMI cases, on the other
hand, yielded lower dollars per case but higher dollars per hour.
However, IRS officials noted that these results were very preliminary
and that several refinements are needed in their methodology for
comparing cases. IRS SB/SE Research staff also acknowledge more data,
including cost data, are needed to conduct a more conclusive cost-
benefit analysis.
Well-Developed Evaluation Plans Increase the Likelihood That
Evaluations Will Yield Methodologically Sound Results and Support
Effective Program and Policy Decisions:
A well-developed and documented evaluation plan can help ensure that
agency evaluations generate performance information needed to make
effective program and policy decisions. Well-developed evaluation plans
include, at a minimum, several key features, such as:
* well-defined, clear, and measurable objectives;
* criteria or standards for determining pilot-program performance;
* clearly articulated methodology, including sound sampling methods,
determination of appropriate sample size for the evaluation design, and
a strategy for comparing the pilot results with other efforts;
* a clear plan that details the type and source of data necessary to
evaluate the pilot, methods for data collection, and the timing and
frequency of data collection; and:
* a detailed data-analysis plan to track the program's performance and
evaluate the final results of the project.[Footnote 11]
In addition, to ensure efficient use of its resources, IRS should
include a cost-effectiveness analysis to ensure that the program
produces sufficient benefits in relation to its costs.
Recognizing the importance of assessing the performance of a new
program, IRS requires that an evaluation plan and report be submitted
with a plan for IRS reorganizations. IRS requires that the evaluation
plan (1) be developed before the proposed reorganization takes place;
(2) include how, when, and by whom the impact of the reorganization
will be evaluated; (3) specify the measures that will be used and the
standards for judging the measures; (4) state how and when the data for
each measure will be obtained and the amount of change that will
indicate success; (5) measure the costs and benefits, including any
savings from the reorganization; (6) establish time lines for the
evaluation; and (7) include the standards for deciding whether to
expand, make permanent, discontinue, or modify the test organization.
The reorganization evaluation report must include a statement of the
business need for the reorganization, results for the measures,
analysis of the results in relation to the stated business need,
conclusions about the impact of the change, and
recommendations.[Footnote 12] The IRS requirements are relevant,
because key features of a sound evaluation plan apply to any program
for which performance information is needed to make effective program
and policy decisions.
IRS Does Not Yet Have a Sound, Documented Evaluation Plan, Which
Undermines Its Ability to Support Management Decisions about the
Productivity of the SRFMI Pilot:
IRS has not yet developed a methodologically sound and documented
evaluation plan for the SRFMI program. IRS's current activities have a
number of limitations.
IRS has not defined clear, measurable objectives with specific
statements of the accomplishments necessary to successfully meet the
objectives. Clear and measurable objectives help ensure that the
appropriate evaluation data are collected and that performance can be
measured against the objectives. IRS has articulated a goal for SRFMI-
-to help reduce the federal tax gap by improving tax compliance. IRS
outlines its basic approach for the SRFMI pilot in a number of briefing
documents and, in its SB/SE FY 2008 and 2009 Plan, alludes to SRFMI in
its discussion of strategies to address the tax gap.[Footnote 13] In
its briefing document for the 2008 Federation of Tax Administrators
conference, IRS indicates SRFMI has potential to reduce the tax gap and
improve voluntary compliance. The slides outline objectives such as:
(1) secure compliance results to drive business decisions, (2) modify
systems and compliance activities to further automate SRFMI, and (3)
reduce the tax gap by making inroads with noncompliant business
entities. These objectives are not clear. For instance, "securing
compliance results to drive business decisions" does not explain what
compliance results would be used or what business decisions the
compliance results would drive. Nor are the objectives measurable. For
example, "reduce the tax gap by making inroads with noncompliant
business entities" does not specify how much of a compliance
improvement would indicate success, or how many business entities or
what dollar amount of new assessments would indicate that IRS is
"making inroads" with noncompliant businesses. IRS officials told us
that individual IRS units using the SRFMI data will have their own
separate objectives for evaluating all data sources for their
compliance activity and that they will measure SRFMI results by the
same standards. They added that IRS is incorporating these objectives
into an overall SRFMI sampling and evaluation plan.
IRS has no criteria or standards for determining where the pilot
program performs adequately to be incorporated into normal IRS
compliance processes. IRS officials said the agency plans to use
research results to help formulate standards but did not provide a
target date when this will occur. IRS officials provided several
projects conducted by SB/SE Research using phase I SRFMI data that will
serve as the basis for developing standards. For example, they gave us
tabulations of results for data matches using federal and Arkansas,
Iowa, and Massachusetts state data and a preliminary analysis of the
usefulness of Colorado and New Jersey tax amnesty data. Such standards
are needed for IRS to determine whether the SRFMI pilot program is
effective.
IRS has not completely articulated its methodology to evaluate the
pilot program. SRFMI methodology should include developing plans for
sound sampling methods, determining appropriate sample sizes, and
comparing the pilot results with IRS's similar ongoing compliance
efforts to determine whether SRFMI can identify cases with higher
noncompliance potential than similar ongoing cases. IRS officials
provided action plans and testing sample plans for individual units
when asked about their methodology. The action plans primarily
contained project-management-type information such as actions or tasks
to be taken by individuals, start date, completion date, and problem
areas rather than a comprehensive description of the methodological
approach for the overall pilot project. The testing sample plans were
questionnaires soliciting information about compliance measures for the
individual unit and the resources available for testing SRFMI data.
None of the plans they presented outline the methods, timing, or
frequency of data collection. Specifying these elements in the plan
would help ensure that adequate, accurate, and timely data will be
available to complete the evaluation. While IRS officials have begun
developing a sampling approach and determining appropriate sample
sizes, IRS officials encountered challenges that have delayed progress.
First, IRS officials told us that information technology support for
identifying cases with higher noncompliance potential than similar
ongoing cases did not occur as planned and, as a result, two compliance
activities--ASFR and Field Examination--performed manual workload
selection on SRFMI cases. Second, IRS is barred by the 3-year statute
of limitations from assessing tax liabilities identified by state tax
data received in the earlier phases of the pilot program.[Footnote 14]
IRS submitted a legislative proposal that would extend the 3-year
limitation for assessments based on information obtained from state and
local tax data.[Footnote 15]
IRS does not have a plan that details the type and source of data
necessary to evaluate the pilot program, methods for data collection,
or the timing and frequency of data collection. The testing sample
plans SB/SE Research has begun developing have information on the type
and source of data it will collect for the four of eight compliance
activities that have filled out the questionnaire. Agency officials
told us that they will ask the rest of the compliance activities to
fill out the same questionnaires.
IRS does not have a data-analysis plan that describes how the data will
be analyzed to track the program's performance and evaluate the final
results. Instead, IRS has analyzed phase I data in an ad hoc manner,
using data from the two compliance activities (i.e., ASFR and Field
Examination) that worked the available cases. A detailed data-analysis
plan is a key feature of an evaluation plan as it sets out who will do
the analysis and when and how data will be analyzed to measure the
pilot program's performance.
IRS does not have a cost-effectiveness-analysis plan to help ensure
that the program produces sufficient benefits in relation to its costs.
For example, SB/SE Research was asked to conduct an analysis with
available phase I matched data using federal tax returns and Arkansas,
Iowa, and Massachusetts state tax-amnesty data to determine SRFMI's
cost effectiveness. The results of the SB/SE Research were not
conclusive, because the data they used were not a representative sample
of ASFR and Field Examination SRFMI cases. Because ASFR and Field
Examination did not have cost data, SB/SE Research could not do a cost-
effectiveness study. Instead, for ASFR, the study compared dollars
assessed per case and dollars assessed per hour worked on SRFMI cases
with normal non-SRFMI cases. For Field Examination, it tracked average
hours worked per return, no-change rate, cycle time, and pick-up rate.
Agency officials told us that IRS has no cost data for the program and
that they do not track the costs of the pilot. IRS acknowledges that
more data are needed to do a complete cost assessment.
Conclusions:
Evaluating the SRFMI pilot can help IRS determine the benefits of using
state taxpayer data for compliance purposes. IRS is exploring new
territory by obtaining and using state taxpayer data to match against
its own to identify taxpayers who do not file federal tax returns or
underreport income on their federal tax returns. However, 2½ years
after the pilot began, as it is entering phase III and is less than 1
year away from a national rollout with 45 states enrolled to deliver
SRFMI files, IRS has yet to develop and document a sound evaluation
plan. The need to evaluate the program is underscored because obtaining
and using SRFMI data imposes costs not only on IRS but also on the
states. Without a sound assessment of pilot program results, IRS may
make poor decisions about the program's future.
Recommendation for Executive Action:
We recommend that the Commissioner of Internal Revenue ensure that IRS
develops an evaluation plan to accurately and reliably assess the SRFMI
pilot program's results. This plan should address all components of the
pilot program and include key evaluation features of a sound plan:
* well-defined, clear, and measurable objectives;
* criteria or standards for determining pilot program performance;
* clearly articulated methodology, including sound sampling methods,
determination of appropriate sample size for the evaluation design, and
a strategy for comparing the pilot results with other efforts; and:
* a clear plan that details the type and source of data necessary to
evaluate the pilot, methods for data collection, and the timing and
frequency of data collection; and:
* a detailed data-analysis plan to track the program's performance and
evaluate the final results of the project.
Agency Comments:
On behalf of the Commissioner of Internal Revenue, the Deputy
Commissioner for Services and Enforcement provided written comments on
a draft of this report in a November 3, 2008, letter. The Deputy
Commissioner agreed it is important to properly document and assess the
SRFMI pilot program as a whole before it is expanded to additional
states. The Deputy Commissioner said a summit will be conducted with
the business unit owners and their respective research functions to
further develop and enhance their existing SRFMI plans to include the
key evaluation features we recommend. In addition, IRS agreed to
develop an overall evaluation plan to accurately and reliably assess
all components of the SRFMI pilot program's results and include the key
evaluation features in our recommendation.
As agreed with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after its date. At that time, we will send copies to the Secretary of
the Treasury, the Commissioner of Internal Revenue, and other
interested parties. This report will also be available at no charge on
GAO's Web site at [hyperlink, http://www.gao.gov]. If you or your staff
have any questions about this report, please contact me at (202) 512-
9110 or brostekm@gao.gov. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last
page of this report. Key contributors to this report are listed in
appendix V.
Signed by:
Michael Brostek:
Director, Tax Issues Strategic Issues Team:
[End of section]
Appendix I: States Participating in Phases I, II, and III:
Below is a list of states participating in the first three phases of
the pilot program. The Internal Revenue Service (IRS) expects to start
a nationwide rollout in July 2009.
Table 1: List of States Participating in State Reverse File Match
Initiative (SRFMI) Pilot Program:
Table 2:
State: Arizona;
Phase I: [Empty];
Phase II: [Empty];
Phase III: x.
State: Arkansas;
Phase I: x;
Phase II: x;
Phase III: x.
State: California;
Phase I: [Empty];
Phase II: x;
Phase III: x.
State: Delaware;
Phase I: [Empty];
Phase II: x;
Phase III: x.
State: Georgia;
Phase I: [Empty];
Phase II: [Empty];
Phase III: x.
State: Iowa;
Phase I: x;
Phase II: x;
Phase III: x.
State: Kansas;
Phase I: [Empty];
Phase II: [Empty];
Phase III: x.
State: Maryland;
Phase I: [Empty];
Phase II: x;
Phase III: x.
State: Massachusetts;
Phase I: x;
Phase II: x;
Phase III: x.
State: New Jersey;
Phase I: [Empty];
Phase II: x;
Phase III: x.
State: New York;
Phase I: [Empty];
Phase II: x;
Phase III: x.
State: Ohio;
Phase I: [Empty];
Phase II: x;
Phase III: x.
State: Washington;
Phase I: [Empty];
Phase II: [Empty];
Phase III: x.
Source: IRS.
Note: Illinois, Louisiana, Nevada, North Dakota, Oregon, Pennsylvania,
South Carolina, and South Dakota may not participate in a nationwide
rollout. New Jersey amnesty data used during phase I were secured
outside of the SRFMI process.
[End of table]
[End of section]
Appendix II: Overview of Data-Matching Process and Further Examinations
or Investigations under SRFMI:
The figure below shows an overview of how the State Reverse File Match
Initiative (SRFMI) data flow between the Internal Revenue Service (IRS)
and state revenue agencies.
Figure 1: Overview of Data Matching under SRFMI Pilot Program:
This figure is a flowchart showing an overview of data matching under
SRFMI pilot program.
[See PDF for image]
Source: GAO analysis of IRS information.
[End of figure]
IRS officials told us that when IRS receives SRFMI data on potentially
noncompliant business and individual taxpayers, IRS conducts data
validation and perfection activities. Next, IRS plans to put the
matched data through a filtering process under development to identify
cases with higher potential for noncompliance and determine how data
should be distributed to the individual units within the Small
Business/Self Employed (SB/SE) Division that will use the data for
compliance activities. IRS officials told us the SRFMI filter will be
continuously refined to take into account law changes, taxpayer
schemes, and other nuances that business units expect to identify as
they develop a history of using the data in compliance activities.
After the filtering process, the SRFMI cases will be distributed to
eight different compliance activities where the business units will
further screen filtered SRFMI data against their program criteria.
Figure 2 shows the distribution of SRFMI data within IRS as well as the
disposition of cases that use SRFMI data.
Figure 2: Overview of the Compliance Activities for Further Examination
or Investigation:
This figure is a chart showing an overview of the compliance activities
for further examination or investigation.
[See PDF for image]
Source: GAO analysis of IRS information.
[End of figure]
[End of section]
Appendix III: Preliminary Compliance Measures by ASFR and Field
Examination Activities:
During 2008, two organization units tracked compliance activities using
phase I State Reverse File Match Initiative (SRFMI) pilot data. The
Automatic Substitute for Return (ASFR) program is a compliance activity
that creates a substitute federal return when none exists. Following
are the traditional compliance measures used by ASFR:
* number of case closures,
* number of taxpayer returns prepared,
* number of defaults,
* number of reconsiderations,
* dollars assessed, and:
* dollars assessed per return.
Small Business/Self-Employed (SB/SE) Field Examination is a compliance
activity during which agents conduct audits in an Internal Revenue
Service (IRS) office or at a taxpayer's place of business. SRFMI cases
are examined by either revenue agents or tax compliance officers. IRS
agents conducting field examinations used federal-state matched SRFMI
data. Field Examination tracked the phase I SRFMI cases it examined
using the measures listed below:
* total closures,
* related return closures,
* average hours per return,
* average dollars per return,
* average dollars per hour,
* total dollars assessed,
* no-change returns,
* no-change rate,
* average closed cycle days, and:
* total nonexamined cases.
[End of section]
Appendix IV: Comments from the Internal Revenue Service:
Department Of The Treasury Internal Revenue Service:
Washington, D.C. 20224:
Deputy Commissioner:
November 3, 2008:
Mr. Michael Brostek:
Director, Tax Issues:
U.S. Government Accountability Office:
441 G Street, N.W.:
Washington, D.C. 20548:
Dear Mr. Brostek:
Thank you for the opportunity to review your draft report entitled "Tax
Administration: IRS Needs to Strengthen Its Approach for Evaluating the
SRFMI Data-Sharing Pilot Program (GAO-09-45)."
We appreciate your taking the time to review the State Reverse File
Matching Initiative (SRFMI) pilot program and agree it is important to
properly document and assess the program as a whole before it is
expanded to additional states. A comprehensive evaluation plan is being
developed that will incorporate the key features you recommend.
As the report acknowledges, the IRS developed action plans and testing
sample plans for each of the business units participating in the SRFMI
pilot program. It was necessary to focus our early efforts on adapting
the wide variety of state data formats to each of our compliance
functions. These formats are frequently mandated by state statutes,
thus preventing the IRS from establishing uniform requirements.
Partnering with state tax agencies provides the IRS with data on
individual and corporate income tax, sales tax, and withholding. By
matching our Master File data against the states' data, we are able to
identify:
* taxpayers who filed a state return but not a federal return;
* differences in income reported on state and federal returns; and:
* taxpayers who filed under state amnesties but failed to file a
federal return. If you have any questions, please contact me or Jeff
Basalla, Acting Director, Campus Compliance Services at (202) 283-7399.
Sincerely,
Signed by:
Linda E. Stiff:
Enclosure:
GAO Recommendation and IRS Responses to GAO Draft Report Tax
Administration: IRS Needs to Strengthen Its Approach for Evaluating the
SRFMI Data-Sharing Pilot Program GAO-09-45
Recommendation: We recommend that the Commissioner of Internal Revenue
ensure that IRS develops an evaluation plan to accurately and reliably
assess the SRFMI pilot-program's results. This plan should address all
components of the pilot program and include key evaluation features of
a sound plan:
* Well-defined, clear, and measurable objectives;
* Criteria or standards for determining pilot program performance;
* Clearly articulated methodology, including sound sampling methods,
determination of appropriate sample size for the evaluation design, and
a strategy for comparing the pilot results with other efforts;
* A clear plan that details the type and source of data necessary to
evaluate the pilot, methods for data collection, and the timing and
frequency of data collection; and:
* A detailed data-analysis plan to track the program's performance and
evaluate the final results of the project.
Comments: A summit will be conducted with the business unit owners and
their respective Research functions to further develop and enhance
their existing SRFMI plans to include the key evaluation features you
recommend in order to commence with data output sample testing. In
addition, we agree to develop an overall evaluation plan to accurately
and reliably assess all components of the SRFMI pilot program's results
and include the key evaluation features cited above.
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Michael Brostek, (202) 512-9110 or brostekm@gao.gov:
Acknowledgments:
In addition to the contact named above, Signora J. May, Assistant
Director; Amy Bowser; Amy Friedlander; Cindy Gilbert; and Lou V. B.
Smith made key contributions to this report.
Footnotes:
[1] GAO, Taxpayer Information: Options Exist to Enable Data Sharing
Between IRS and USCIS but Each Presents Challenges, [hyperlink,
http://www.gao.gov/products/GAO-06-100] (Washington, D.C.: Oct. 11,
2005).
[2] Ibid. While IRS is generally prohibited from disclosing taxpayer
information, IRS is authorized to share taxpayer data with state tax
officials under 26 U.S.C. § 6103(d) for tax-administration purposes.
[3] P.H. Rossi, M.W. Lipsey, and H.E. Freeman, Evaluation: A Systematic
Approach (Thousand Oaks, Calif.: 2004); GAO, Designing Evaluations,
[hyperlink, http://www.gao.gov/products/GAO/PEMD-10.1.4] (Washington,
D.C.: May 1991); GAO, Assessing Social Program Impact Evaluations: A
Checklist Approach, [hyperlink, http://www.gao.gov/products/PAD-79-1
(Washington, D.C.: October 1978); B.R. Worthen, J.R. Sanders, and J.L.
Fitzpatrick, Program Evaluation: Alternative Approaches and Practical
Guidelines (New York, N.Y.: 1997).
[4] For purposes of the GLDEP, a state is defined as any of the 50
states, the District of Columbia, certain U.S. possessions or
territories, and any municipality with a population in excess of
250,000 that imposes a tax on income or wages (26 U.S.C. § 6103(b)(5)).
[5] An IRS Form 1099-MISC is used to report payments of $600 or more
for services performed for a trade or business by a nonemployee,
including independent contractors.
[6] In the context of SRFMI, these cases are called "matches."
[7] IRS has requested that states provide individual, corporate, sales,
and withholding data for SRFMI, but participating states have not
always been able to provide all four.
[8] A data extract is a computer-generated file that contains specific
data elements.
[9] Validation is the process of verifying that IRS recognizes the
taxpayer using two data elements--Taxpayer Identification Number (TIN)
and taxpayer name. Perfection is the process of matching the taxpayer
TIN and name when the two almost match. IRS makes small adjustments to
either data element to perfect the record. When IRS receives state data
for SRFMI, it first checks to ensure the file structure meets SRFMI
specifications and then validates and perfects the data.
[10] Tax amnesty programs are held to collect taxes owed from prior
years and to place those who previously avoided taxation on the tax
rolls.
[11] P.H. Rossi, M.W. Lipsey, and H.E. Freeman, Evaluation; [hyperlink,
http://www.gao.gov/products/GAO/PEMD-10.1.4]; GAO, [hyperlink,
http://www.gao.gov/products/PAD-79-1]; B.R. Worthen, J. R. Sanders, and
J.L. Fitzpatrick, Program Evaluation.
[12] IRS, Internal Revenue Manual 1.1.4 --Organizational Planning,
Exhibit 1.1.4-6--Evaluation Plan and Report Requirements for IRS
Reorganizations [hyperlink, http://www.irs.gov/irm/part1/ch01s03.html],
accessed Jun. 3, 2008.
[13] The SB/SE FY 2008 and 2009 Plan presents SB/SE's efforts to
identify and address the most critical strategic issues facing SB/SE.
[14] Section 6501(a) of the Internal Revenue Code requires IRS to
assess additional federal tax liabilities in the form of tax, interest,
penalties, and additions to tax within 3 years after the date a return
is filed. After the required time, IRS cannot generally assess the
additional liabilities. If the additional tax liabilities are assessed
within the allowable time frame, IRS is then limited to 10 years to
collect the amounts assessed. 26 U.S.C. § 6502(a).
[15] IRS recently submitted a revenue proposal providing an exception
to the 3-year statute of limitations for assessment of federal tax
liability resulting from adjustments to state or local tax liability.
This proposal extends the statute of limitations by 1 year from the
date the taxpayer files an amended tax return with IRS reflecting
adjustments to the state or local tax return or by 2 years from the
date IRS receives information from the state or local revenue agency
under an information-sharing agreement. U.S. Department of the
Treasury, General Explanations of the Administration's Fiscal Year 2009
Revenue Proposals (Washington, D.C.: February 2008).
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
