Tax Administration
IRS Should Reassess the Level of Resources for Testing Forms and Instructions Gao ID: GAO-03-486 April 11, 2003Taxpayers rated the Internal Revenue Service's (IRS) ability to provide clear and easy-to-use forms and instructions among the lowest of 27 indicators of service in 1993. Due to continuing concerns about unclear forms and instructions, GAO was asked to determine (1) whether and how often IRS tests the clarity of new and revised individual income tax forms and instructions; (2) the benefits, if any, of testing forms and instructions for clarity prior to their use; and (3) whether any factors limit IRS's ability to do more tests and if so, how they can be addressed.
IRS used taxpayers and its employees to test revisions to five individual income tax forms and instructions from July 1997 through June 2002. According to IRS officials, they revised about 450 tax forms and instructions in 2001, many of which were for individual income tax returns. Testing forms and instructions can help ensure their clarity and thereby benefit taxpayers and IRS by, for instance, reducing taxpayers' time to understand and complete tax forms, reducing calls to IRS for assistance, and reducing taxpayer errors. Due to similar benefits, federal agencies we contacted that routinely collect information from the public test their questionnaires. Quantifying benefits due to testing is difficult, but IRS's experience in revising and testing Earned Income Credit and Child Tax Credit forms and instructions suggests that benefits of testing in some cases can considerably exceed the cost of testing. If taxpayers who did their own tax returns needed 1 less minute to understand these two credits due to testing, their time saved, valued at the minimum wage, would be worth $1.2 million; IRS's contracting cost for the two tests was $56,000. Although IRS officials recognized that testing could be beneficial, they cited tight time frames and constrained resources as limiting their ability to do more tests. While IRS faces time constraints when making some changes to forms and instructions due to the passage of new laws, not all changes are time constrained. IRS does not have procedures specifying which versions of draft forms and instructions should be tested with taxpayers or when in its annual forms development process testing should occur. Resources currently available for testing are limited but the office responsible for testing has not developed data on missed testing opportunities and has limited data on the benefits that have been realized when testing occurred. IRS's planning and budgeting process uses such data to support resource allocation decisions.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-486, Tax Administration: IRS Should Reassess the Level of Resources for Testing Forms and Instructions
This is the accessible text file for GAO report number GAO-03-486
entitled 'Tax Administration: IRS Should Reassess the Level of
Resources for Testing Forms and Instructions' which was released on
April 11, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products‘ accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
Report to the Chairman and Ranking Minority Member, Committee on
Finance, U.S. Senate:
United States General Accounting Office:
GAO:
April 2003:
TAX ADMINISTRATION:
IRS Should Reassess the Level of Resources for Testing Forms and
Instructions:
GAO-03-486:
GAO Highlights:
Highlights of GAO-03-486, a report to Chairman and Ranking Member of
the Senate Committee on Finance
Why GAO Did This Study:
Taxpayers rated the Internal Revenue Service‘s (IRS) ability to provide
clear and easy-to-use forms and instructions among the lowest of 27
indicators of service in 1993. Due to continuing concerns about
unclear forms and instructions, GAO was asked to determine
* whether and how often IRS tests the clarity of new and revised
individual income tax forms and instructions;
* the benefits, if any, of testing forms and instructions for clarity
prior to their use; and
whether any factors limit IRS‘s ability to do more tests and if so, how
they can be addressed.
What GAO Found:
IRS used taxpayers and its employees to test revisions to five
individual income tax forms and instructions from July 1997 through
June 2002. According to IRS officials, they revised about 450 tax forms
and instructions in 2001, many of which were for individual income tax
returns.
Testing forms and instructions can help ensure their clarity and
thereby benefit taxpayers and IRS by, for instance, reducing taxpayers‘
time to understand and complete tax forms, reducing calls to IRS for
assistance, and reducing taxpayer errors. Due to similar benefits,
federal agencies we contacted that routinely collect information from
the public test their questionnaires. Quantifying benefits due to
testing is difficult, but IRS‘s experience in revising and testing
Earned Income Credit and Child Tax Credit forms and instructions
suggests that benefits of testing in some cases can considerably exceed
the cost of testing. If taxpayers who did their own tax returns needed
1 less minute to understand these two credits due to testing, their
time saved, valued at the minimum wage, would be worth $1.2 million;
IRS‘s contracting cost for the two tests was $56,000.
Although IRS officials recognized that testing could be beneficial,
they cited tight time frames and constrained resources as limiting
their ability to do more tests. While IRS faces time constraints when
making some changes to forms and instructions due to the passage of new
laws, not all changes are time constrained. IRS does not have
procedures specifying which versions of draft forms and instructions
should be tested with taxpayers or when in its annual forms development
process testing should occur. Resources currently available for testing
are limited but the office responsible for testing has not developed
data on missed testing opportunities and has limited data on the
benefits that have been realized when testing occurred. IRS‘s planning
and budgeting process uses such data to support resource allocation
decisions.
What GAO Recommends:
This report makes a series of recommendations that, if fully
implemented, would improve IRS‘s annual process for creating and
revising individual income tax forms and instructions by helping to
ensure that its scarce testing resources are consistently applied to
the highest priorities, impediments to testing are lessened, and
appropriate information is developed so IRS management can better
ensure that adequate resources are available to support testing in view
of the potential benefits to taxpayers and IRS.
IRS agreed with our recommendations and plans to implement all but one
of them in time for the 2004 forms development cycle.
www.gao.gov/cgi-bin/getrpt?GAO-03-486.
To view the full report, including the scope
and methodology, click on the link above.
For more information, contact Michael Brostek at (202) 512-9110 or
brostekm@gao.gov.
Contents:
Letter:
Results in Brief:
Background:
IRS Tested Five Tax Forms and Instructions Over 5 Years:
Testing Written Documents Helps Ensure Clarity and Benefits Taxpayers
and IRS:
IRS Has Not Addressed Constraints to Increased Testing:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Use of Testing in NCHS, Census, and BLS:
Appendix III: Comments from the Internal Revenue Service:
Tables:
Table 1: Forms and Instructions IRS Tested Using Individual Taxpayers
Between July 1997 and June 2002:
Table 2: Percentage Decrease in the Number of EIC and Child Tax Credit
Errors before and after Revising and Testing:
Table 3: Potential Benefits if Testing Helps Ensure the Clarity of Tax
Forms and Instructions:
Table 4: Illustration of Potential Savings to IRS Alone in 2000 from
Testing EIC and Child Tax Credit Forms and Instructions:
Table 5: Illustration of Potential Savings to Taxpayers and IRS in 2000
from Error Reductions Alone from Testing EIC and Child Tax Credit Forms
and Instructions:
Table 6: Illustration of Potential Benefit to Taxpayers in 2000 if
Testing Reduced the Time Needed to Understand and Complete EIC and
Child Tax Credit Forms:
Table 7: Illustration of Potential Cost Savings to IRS in 2002 if the
Rate Reduction Credit Instructions Had Been Tested and Errors Were
Reduced:
Table 8: Illustration of Potential Savings to Taxpayers and IRS in 2002
from Potential Error Reductions if Testing the Rate Reduction Credit
Had Reduced Taxpayer Errors:
Table 9: Illustration of Potential Benefit to Taxpayers in 2002 if
Testing Reduced the Time Needed to Understand and Complete the Rate
Reduction Credit:
Figures:
Figure 1: Simplified Overview of IRS‘s Annual Tax Forms Development
Process:
Figure 2: Simplified Overview of Where Testing Can Occur in IRS‘s
Annual Tax Forms Development Process:
This is a work of the U.S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO‘s product.
Abbreviations:
BLS: Bureau of Labor Statistics:
EIC: Earned Income Credit:
ERS: Error Resolution System:
FTE: full-time equivalent:
IRS: Internal Revenue Service:
NCHS: National Center for Health Statistics:
OMB: Office of Management and Budget:
TIGTA: Treasury Inspector General for Tax Administration:
W&I: Wage and Investment:
United States General Accounting Office:
Washington, DC 20548:
April 11, 2003:
The Honorable Charles E. Grassley
Chairman,
Committee on Finance
United States Senate:
The Honorable Max Baucus
Ranking Minority Member,
Committee on Finance
United States Senate:
Internal Revenue Service (IRS) tax forms and instructions that are not
clear and understandable have been a long-standing frustration of
taxpayers.[Footnote 1] For example, respondents to a 1993 customer
satisfaction survey rated IRS‘s ability to provide clear and easy-to-
use tax forms and instructions and to minimize taxpayer‘s burden as the
lowest of 27 indicators of service.[Footnote 2] More recently, we cited
the instructions for the tax year 2001 rate reduction credit as
confusing and one factor contributing to millions of taxpayer
errors.[Footnote 3]
To be fair, IRS faces a challenge in clearly communicating with
taxpayers due to the complexity of the Internal Revenue Code. However
the code‘s complexity also underscores the importance of tax forms and
instructions that are as clear and understandable as possible.
One approach with recognized potential for aiding in the development of
clearer forms and instructions is testing. By testing draft forms and
instructions with taxpayers, IRS might be able to detect and revise
language that is unnecessarily confusing. The Treasury Inspector
General for Tax Administration (TIGTA) and we have both previously
recommended that IRS do more to, for instance, identify what individual
taxpayers find difficult to understand about tax forms or
publications.[Footnote 4]
Because of your interest in ensuring that taxpayers have clearer,
understandable tax forms and instructions and because of the potential
for testing to contributing to that end, you asked us to determine:
* whether and how often IRS used taxpayers to test the clarity of new
or revised individual income tax forms and instructions;
* the benefits, if any, of having taxpayers test forms and instructions
for clarity prior to their use by the public; and
* whether any factors limit IRS‘s ability to test more forms and
instructions for clarity and if so, how these factors can be addressed.
To address these objectives, we interviewed officials (1) in IRS‘s Wage
and Investment (W&I) Division‘s Tax Forms and Publications Division,
(2) from three federal agencies that use private citizens extensively
in testing research and data collection activities, and (3) a private
research firm that specializes in testing written documents, such as
forms and surveys, for a variety of public and private sector clients.
We also developed illustrations of the potential benefits to taxpayers
and IRS of testing forms and instructions for clarity. Our scope and
methodology are discussed in greater detail in appendix I to this
report.
Results in Brief:
From July 1997 through June 2002, IRS used taxpayers or its employees,
primarily in focus groups, to test 5 individual income tax forms or
instructions. According to IRS officials, they revised about 450 forms
and instructions in 2001, many of which were for individual income tax
returns.
Testing written documents, such as forms and instructions, prior to
their use helps ensure that they communicate clearly. Testing thereby
helps (1) lessen the burden to respondents of understanding and
completing forms and (2) reduce errors made by respondents. To the
extent that taxpayers find IRS‘s tax forms and instructions that were
tested easier to understand, IRS is less likely to be contacted by
taxpayers for assistance in fulfilling their tax obligations and is
likely to have fewer taxpayer errors to detect and correct. Benefits
like these may be difficult to quantify, but form the basis for several
agencies we contacted, as well as ourselves, to make testing a standard
practice when developing or modifying documents used to collect
information from the public. IRS‘s recent, limited experience with
testing forms and instructions used by individuals suggests that
testing is beneficial. At least in some cases the potential benefits of
testing IRS‘s forms and instructions are likely to be substantially
greater than the costs of testing to IRS.
Although IRS officials said that making greater use of testing to
improve clarity of forms and instructions could be beneficial, they
cited tight time frames for making changes and limited resources in the
Tax Forms and Publications Division as the primary factors that limit
their ability to conduct more tests. Although IRS does face time
constraints when making some changes, especially when changes are due
to new tax laws, time constraints do not apply to all changes IRS makes
to forms and instructions for individual taxpayers. Further, IRS does
not have clear procedures specifying which draft versions of forms and
instructions should be tested with taxpayers or when testing should
occur during the annual forms update process. Resources available
within the division responsible for developing forms and instructions
are limited. For example, one employee has responsibility for
organizing testing efforts. However, the division has not developed
information that IRS uses when making resource allocation decisions.
The office has not developed data on the universe of changes for which
testing is most likely to yield significant benefits to taxpayers and
IRS and it has only limited data on the benefits that have been
realized when testing has occurred.
This report makes a series of recommendations to improve IRS‘s annual
process for creating and revising individual income tax forms and
instructions through ensuring that (1) its scarce testing resources are
consistently applied to the highest priorities, (2) impediments to
testing are lessened, and (3) appropriate information is developed so
IRS managers can ensure that adequate resources are available for
testing in view of the potential benefits to taxpayers and IRS. IRS
agreed with our recommendations and plans to implement all but one of
them in time for the 2004 forms development cycle. The remaining
recommendation--to ensure that an appropriate range of evaluations is
conducted of tests--would take more time to put into practice.
Background:
Although most organizational components within IRS are involved, the
Tax Forms and Publications Division within IRS‘s Media and Publications
Division of the W&I Division is primarily responsible for creating and
improving tax forms, instructions, and other documents. One goal of the
Tax Forms and Publications Division is to make tax forms and
instructions as clear and understandable as possible. It is divided
into three branches--Individual Forms and Publications, Business Forms
and Publications, and Tax Exempt/Government Entities and Specialty
Forms and Publications. As of late January 2003, 103 persons were
assigned to the Tax Forms and Publication Division, including about 15
persons whose primary responsibility was creating, revising, and
reviewing individual income tax forms and instructions.
Many tax forms and instructions are revised annually, often with short
turnaround times and in response to tax law changes. IRS also
periodically reviews tax forms and, when appropriate, schedules them
for revision. According to IRS‘s estimate, it revised about 450 tax
forms and instructions in 2001 that affected individual and business
tax returns. In addition to tax law changes, revisions to tax forms and
instructions generally reflected procedural changes, legal rulings, and
feedback from internal and external stakeholders about the
understandability of forms and instructions.
As illustrated in figure 1, the annual tax forms development process
generally starts with a review of the current tax forms. IRS‘s tax law
specialists review the existing forms and instructions to determine
what changes, if any, may be needed to reflect tax law changes and
other requirements. The tax law specialists consider comments from a
variety of sources both within and outside IRS. For example, comments
may be obtained from IRS customer service staff in toll-free call
centers who answer calls from taxpayers and have firsthand knowledge of
particular forms or instructions that were confusing to taxpayers.
IRS‘s Taxpayer Advocate Service staff may also provide comments useful
to the tax law specialists. The Tax Forms Coordinating Committee,
comprised of representatives from all of IRS‘s key components, the
Department of the Treasury, and IRS‘s Chief Counsel, reviews draft
forms to help ensure that they are not overly burdensome and that they
conform to legal and technical requirements. Draft forms are generally
posted to IRS‘s external website so that external stakeholders and
taxpayers may review and comment on them.
The Office of Management and Budget (OMB) is responsible for approving
each form once every 3 years. The purpose of OMB‘s approval is to
assess IRS‘s compliance with the Paperwork Reduction Act, which, among
other things, requires agencies to assess the extent of burden the
information they collect imposes on the public. Under the Paperwork
Reduction Act, OMB must approve new forms and major revisions to
existing ones. After OMB‘s approval, the forms and instructions are
sent to IRS‘s vendors to be printed. Generally, IRS needs to have
approved forms ready for printing by early October to ensure that they
can be printed and distributed to the public when the tax filing season
starts the following January.
Figure 1: Simplified Overview of IRS‘s Annual Tax Forms Development
Process:
[See PDF for image]
Note: GAO analysis of IRS data.
[End of figure]
IRS Tested Five Tax Forms and Instructions Over 5 Years:
As shown in table 1, IRS used taxpayers or IRS employees to test the
clarity of five individual income tax forms and instructions from July
1997 through June 2002.[Footnote 5] IRS relied primarily on focus
groups to do the testing. Contractors using private citizens did three
of the tests (i.e., Earned Income instructions, Child Tax Credit
instructions, and Schedule D) and IRS did the other two using IRS
employees.
Table 1: Forms and Instructions IRS Tested Using Individual Taxpayers
Between July 1997 and June 2002:
Forms and instructions: Earned Income Credit schedule; Year tested:
1998; Number of participants: 50; Testing method: 5 focus groups.
Forms and instructions: Earned Income Credit instructions; Year tested:
1999; Number of participants: 104; Testing method: 10 focus groups.
Forms and instructions: Child Tax Credit instructions; Year tested:
1999; Number of participants: 104; Testing method: 10 focus groups.
Forms and instructions: Schedule D Form-Capital Gains and Losses; Year
tested: 2001; Number of participants: 48; Testing method: 6 focus
groups; 8 one-on-one interviews.
Forms and instructions: Innocent Spouse Application form; Year tested:
2002; Number of participants: 52; Testing method: 6 focus groups.
Source: IRS.
Note: Forms and instructions may also refer to schedules and
worksheets.
[End of table]:
The two testing methods that IRS used--focus groups and one-on-one
interviews--are among the commonly used methods to obtain data from
individuals on whether documents such as forms and instructions are
clear and understandable. Focus groups generally consist of a small
number of participants--about 8 to 12 persons---and are usually
selected and organized around the focus group topic. Focus groups are a
form of group interviewing that relies on interaction within the group
to obtain the impressions of a group of people but not necessarily the
impressions of each participant. One-on-one interviews, which aim to
obtain individual attitudes, beliefs, and feelings, are used to probe
individuals about specific difficulties they may have with completing a
form or reading instructions. In some instances, these methods may be
used in combination depending upon the particular circumstances of the
test. As noted in table 1, IRS used both focus groups and one-on-one
interviews in testing the Schedule D form.
Testing Written Documents Helps Ensure Clarity and Benefits Taxpayers
and IRS:
Testing written documents such as tax forms and instructions helps
ensure they are clear, thereby benefiting taxpayers and IRS.
Researchers from three federal agencies and a private research firm
said testing leads to clearer documents as well as more accurate
responses. Our guidance on developing and using questionnaires also
recommends testing prior to distribution.[Footnote 6] IRS‘s experience
indicates that testing likely improves the clarity of tax forms and
instructions and may therefore help reduce the number of errors
taxpayers make. Although limited data were available on the costs and
benefits of testing IRS forms and instructions, recent changes to
Earned Income Credit (EIC) and Child Tax Credit forms illustrate the
potential for benefits to significantly exceed the costs of testing.
Experiences of Three Federal Agencies, a Private Research Firm, and IRS
Support Use of Testing:
Researchers from the National Center for Health Statistics (NCHS), the
Census Bureau, and the Bureau of Labor Statistics (BLS) said that
testing helps ensure that their documents are clear, and therefore
users are more likely to understand them and complete them correctly.
Consequently, researchers from the agencies said they routinely test
written documents, such as forms or surveys, prior to public
distribution. A representative of a private research firm also said
that testing ensures that documents communicate clearly and that his
firm and many others perform such testing for a wide variety of private
and public clients.
The researchers from the federal agencies told us that the benefits of
testing are difficult to quantify and, in some instances, may not be
quantifiable. NCHS officials stated that the experience they have
gained over time from testing and revising documents has helped them
develop clearer forms from the outset, a benefit that may not be
quantifiable. A Census researcher also agreed that testing is
beneficial but difficult to quantify. Nevertheless, the Census
researcher said that testing documents prior to public use helps ensure
that they are clear and understandable, which gives the agency a
greater chance of receiving accurate responses and which lessens the
need for follow-up interviews. Similarly, a BLS researcher noted that,
though difficult to quantify, testing benefits the agency by reducing
errors made by respondents and those reductions can result in savings
of time, money, and effort for BLS. See appendix II for additional
information on the use of testing in these three agencies.
The representative of the private research firm also provided some
perspective of its experiences in conducting tests for public and
private sector customers. This firm, like many similar companies,
provides a variety of testing and data collection services to public
and private sector customers. The firm arranges and conducts focus
groups, one-on-one interviews, and other tests in order to ensure the
clarity of forms, instruction manuals, surveys, and Web sites among
other things. Focus groups generally involve 12 participants and cost
around $6,700 to $7,600, excluding costs to develop the item to be
tested but including incentive pay for participants that could range
from $25 to $50 per participant. These costs may be higher, the
representative said, if the participants come from special groups. For
example, if the participants are medical doctors, incentive pay could
be as high as $250 per participant.
The researcher also said that private firms vary in how much they spend
to test their documents. He estimated that firms generally spend
between $300,000 and $500,000 to ensure that a form or document is
clear and will meet the firm‘s needs. In some cases, firms spend more
than $500,000 to do a series of tests, making revisions between each
test, before they arrive at a final version of a form or document. He
also added that most firms do not spend at levels that would allow them
to test all their forms and documents.
While maintaining that testing is beneficial, researchers also stated
that testing is not fail-safe. It can help identify particular parts of
a form that are not clear, researchers said, but it cannot ensure that
subsequent changes to the form will entirely resolve the clarity issue.
In addition, testing may identify problems participants have in
completing written documents, but the participants‘ problems may be
related to other issues, such as poor math skills, rather than
confusing or unclear documents.
Testing questionnaires before distribution is also recommended as a
quality assurance measure in our guidance on developing and using
questionnaires. According to our guidance, testing questionnaires
before they are used is one of the best ways to ensure that the
document actually communicates what it was intended to communicate and
that users will uniformly interpret it. Testing increases the
likelihood that respondents will provide the information needed and
helps to alleviate inaccurate responses. Our guidance is also
consistent with professional literature on survey design. According to
professional literature, ’reducing measurement error through better
question design is one of the least costly ways to improve survey
estimates. For any survey, it is important to attend to careful
question design and pretesting.“[Footnote 7]
IRS‘s recent limited experience with testing indicates that testing may
help ensure the clarity of tax forms and instructions. In 1999, IRS
revised the forms and instructions related to the EIC and the Child Tax
Credit, tested the revised forms and instructions, and revised them
again based on the test results. The following year, when the revised
and tested forms and instructions were used by taxpayers, the error
rates for the EIC and the Child Tax Credit decreased by 28 and 35
percent, respectively, as shown in table 2.[Footnote 8]
Table 2: Percentage Decrease in the Number of EIC and Child Tax Credit
Errors before and after Revising and Testing:
Forms/Instructions: Earned Income Credit; Number of errors on tax
returns filed in 1999 (before testing): 1,797,162; Number of errors on
tax returns filed in 2000 (after testing): 1,296,095; Percentage
decrease in errors from
1999 to 2000: 28.
Forms/Instructions: Child Tax Credit; Number of errors on tax returns
filed in 1999 (before testing): 1,430,394; Number of errors on tax
returns filed in 2000 (after testing): 934,289; Percentage decrease in
errors from
1999 to 2000: 35.
Source: IRS data.
[End of table]:
IRS officials told us they attribute part of the decrease in EIC errors
to a new approach officials developed for structuring EIC forms and
instructions and part to the improvements in the draft documents that
resulted from testing the revised forms and instructions. Before the
EIC forms and instructions were revised, the instructions included a
definition and example describing a qualifying child that taxpayers had
to interpret.[Footnote 9] Incorrectly claimed qualifying children have
been a major source of EIC errors. IRS revised the instructions so that
taxpayers would answer a series of ’yes/no“ questions to determine if
they have a qualifying child instead of relying on their
interpretations of the definition or example of a qualifying child.
They then tested the old format and the new format. The number of
errors decreased substantially when taxpayers used the new format. IRS
then made some final changes to clarify the instructions based on test
results. In IRS officials‘ opinions, the ’yes/no“ format made it
clearer for taxpayers to determine if they had a qualifying child.
Benefits of Testing Forms and Instructions Can Far Exceed IRS‘s Costs:
The benefits of testing some changes to IRS‘s forms and instructions
can considerably exceed IRS‘s costs to do tests, especially because so
many taxpayers can be affected by improvements in clarity that may
result from testing. IRS‘s contract costs, including travel, for
testing changes to the EIC and Child Tax Credit forms and instructions
were about $56,000 and these costs may have been offset within IRS
alone in the year that the change was implemented. More significantly,
if testing changes to forms and instructions for these credits led to a
1-minute reduction, on average, in the time taxpayers needed to
understand and complete the forms during the 2000 tax filing season,
affected taxpayers would have saved 240,000 hours worth $1.2 million
valued at minimum wage.
Taxpayers and IRS Can Realize a Wide Range of Benefits and Costs From
Testing:
Testing has the potential to yield a wide range of benefits to
taxpayers and IRS. Table 3 summarizes some of the potential benefits
that could result if testing helps clarify tax forms and instructions.
If the form or instruction that has been revised and tested remains
unchanged, some potential benefits could recur annually for the life of
the form or instruction.
Table 3: Potential Benefits if Testing Helps Ensure the Clarity of Tax
Forms and Instructions:
IRS‘s benefits:
* Fewer errors that need to be corrected before
processing returns;
* Reduction in audits due to clarity-related taxpayer errors and
redirection of audit resources to other noncompliant taxpayers;
* Reduction in demand for taxpayer assistance at local IRS offices
may allow IRS to provide better service to other taxpayers;
* Reduction in demand for assistance via IRS‘s toll-free telephone
service related to clarity issues may enable IRS to answer some calls
that would otherwise have not been answered;
* Experience and knowledge gained through testing may lead to
creation of clearer forms and instructions in the future; Taxpayers‘
benefits:
* Increased clarity of IRS‘s forms and instructions reduces;
-time to complete tax forms;
-expenses associated with obtaining assistance in completing tax forms;
* Reduced burden from not having to deal with IRS notices, such as
reduced time to;
-open and read IRS‘s notices;
-decide what to do;
-research tax records;
-prepare response to IRS;
-copy and mail response, if necessary, to IRS;
-call IRS for assistance.
Source: GAO.
:
[End of table]
From the taxpayer‘s perspective, benefits from testing could include
avoiding the burdens associated with (1) interacting with IRS if they
make mistakes due to unclear forms and instructions and (2)
understanding and complying with unclear forms and instructions. From
IRS‘s perspective, benefits are generally in the form of opportunities
to use its resources better serving other taxpayers and enforcing the
tax laws.[Footnote 10]
Because IRS makes many changes to forms and instructions affecting
individual taxpayers every year, ranging from very simple to more
complex changes, the benefits can vary according to the type of changes
made. Some forms or instructions may change simply to update certain
dollar thresholds based on inflation and these changes may be unlikely
to be confusing or unclear to taxpayers. However, in other cases
changes may introduce new requirements or concepts to taxpayers, such
as when new rules are established through legislation or regulation.
Changes intended to address situations like these may be more likely to
be confusing or unclear to taxpayers, which could result in a burden on
taxpayers to understand their obligations in preparing their tax forms
and, possibly, to errors that lead to subsequent interactions with IRS
to correct their returns.
To the extent that a form or instruction is unclear and the lack of
clarity leads to taxpayer errors, the method IRS uses to detect the
errors can affect the costs IRS incurs as a consequence. If a
taxpayer‘s error can be detected by IRS and corrected under its ’math
error“ procedures, which rely extensively on automated processes, the
cost to IRS to correct the error is likely to be small.[Footnote 11] On
the other hand, if unclear forms or instructions lead to compliance
errors that are detected and addressed through audits conducted through
the mail, in IRS offices, or in the taxpayer‘s location, the costs to
IRS are likely to be higher in part because these processes are more
labor intensive.
The burden and costs taxpayers might avoid if testing helps clarify
forms and instructions, and thereby helps taxpayers avoid errors, can
vary substantially just as IRS‘s costs can vary. In general, because
taxpayers need only respond if they disagree with an IRS notice stating
that it has corrected an error under its math error procedures, the
taxpayer‘s burden and cost are likely to be lower than if IRS contacts
the taxpayer as part of an audit since audits require taxpayer
responses and reviews of taxpayers‘ books and records.[Footnote 12]
Testing Benefits Can Far Exceed IRS‘s Costs:
Illustrations we developed of the potential benefits and costs of
testing forms and instructions show that at least in some cases
benefits can be substantially greater than the costs to IRS to do
tests. The benefits of testing to IRS alone can potentially exceed its
testing costs in the first year a change is implemented. But, primarily
because a small change in the time required of taxpayers to understand
their tax obligations can total to a large aggregate benefit, taking
taxpayers‘ benefits into account can yield total benefits substantially
above IRS‘s costs.
IRS officials have not attempted to develop quantitative estimates of
the benefits to taxpayers and IRS that may result from testing forms
and instructions and the costs IRS incurs to achieve those benefits.
IRS officials did believe that because taxpayers made fewer errors when
using the revised EIC and the Child Tax Credit forms and instructions
as shown in table 2, IRS spent less time and money correcting errors
related to them.[Footnote 13] The officials said they could not
quantify the cost savings because IRS does not track error correction
costs by type of error.
To provide some perspective on the potential magnitude of benefits and
costs that may be realized due to testing changes to forms and
instructions, we analyzed the changes IRS made to EIC and Child Tax
Credit forms and instructions. Our analyses are illustrations and not
actual assessments of benefits and costs that were associated with
testing these forms and instructions because complete data were not
available on the potential benefits and costs. Further, in constructing
our illustrations we sought to be conservative in estimating benefits,
in part because we did not have information on the full range of costs
IRS incurred to test forms and instructions. Our illustrations focus on
(1) a narrow set of benefits to IRS alone due to potential reductions
in taxpayer errors, (2) those benefits plus certain benefits to the
taxpayers from reduced errors, and (3) potential benefits to taxpayers
in reduced time to do their taxes. See appendix I for details on the
methodology we used in developing our illustrations.
Our first illustration quantifies a narrow set of benefits to IRS alone
from testing EIC and Child Tax Credit forms and instructions--that is,
the benefits IRS may have realized due to reduced numbers of errors
that are handled under its math error procedures. It is likely that to
the extent testing contributed to better taxpayer understanding of
these two credits, IRS would have obtained other benefits. For
instance, because improperly claimed qualifying children is one of the
leading causes of the EIC‘s high noncompliance rate,[Footnote 14] if
clarified EIC forms and instructions lead fewer taxpayers to improperly
claim the EIC, IRS would likely be able to free some of its EIC-related
audit resources for other audits or to audit EIC returns that it might
otherwise have had insufficient resources to cover. In fiscal year
2002, IRS used about 1,400 full-time equivalent (FTE) staff
years[Footnote 15] for correspondence audits of EIC issues.
Our analysis in table 4 shows the amount by which IRS‘s potential cost
savings from not having to correct EIC and Child Tax Credit errors may
have exceeded its testing costs given differing assumptions about how
much testing alone may have contributed to reduced taxpayer errors. As
illustrated, IRS would have saved more in cost avoidance (thereby
freeing resources to work elsewhere) in the first year of the change
alone than it spent on the contracted testing of the forms and
instructions if half of the reduction in errors was due to
testing.[Footnote 16] If only 10 or 25 percent of the reduction was due
to testing, then IRS would not have saved more than it spent on the
testing contract in the first year. However, some of the benefits of a
change in forms or instructions continue to be realized in future
years. Again, the illustration does not consider other benefits IRS may
have realized.
Table 4: Illustration of Potential Savings to IRS Alone in 2000 from
Testing EIC and Child Tax Credit Forms and Instructions:
Form and instruction: EIC; Differing assumed levels of errors avoided:
50%; Assumed number of errors eliminated due to testing: 250,534;
Estimated direct labor cost to correct an error[A]: $0.36; Estimated
cost to correct eliminated errors: $90,000; Potential IRS cost savings
assuming $56,000 cost to perform test[B]: $34,000.
Differing assumed levels of errors avoided: 25%; Assumed number of
errors eliminated due to testing: 125,267; Estimated direct labor cost
to correct an error[A]: 0.36; Estimated cost to correct eliminated
errors: 45,000; Potential IRS cost savings assuming $56,000 cost to
perform test[B]: (11,000).
Differing assumed levels of errors avoided: 10%; Assumed number of
errors eliminated due to testing: 50,107; Estimated direct labor cost
to correct an error[A]: 0.36; Estimated cost to correct eliminated
errors: 18,000; Potential IRS cost savings assuming $56,000 cost to
perform test[B]: (38,000).
Form and instruction: Child tax credit; Differing assumed levels of
errors avoided: 50%; Assumed number of errors eliminated due to
testing: 248,053; Estimated direct labor cost to correct an error[A]:
0.36; Estimated cost to correct eliminated errors: 89,000; Potential
IRS cost savings assuming $56,000 cost to perform test[B]: 33,000.
Differing assumed levels of errors avoided: 25%; Assumed number of
errors eliminated due to testing: 124,026; Estimated direct labor cost
to correct an error[A]: 0.36; Estimated cost to correct eliminated
errors: 45,000; Potential IRS cost savings assuming $56,000 cost to
perform test[B]: (11,000).
Differing assumed levels of errors avoided: 10%; Assumed number of
errors eliminated due to testing: 49,611; Estimated direct labor cost
to correct an error[A]: 0.36; Estimated cost to correct eliminated
errors: 18,000; Potential IRS cost savings assuming $56,000 cost to
perform test[B]: (38,000).
Source: IRS and GAO.
Note: GAO analysis of IRS data.
[A] IRS officials provided (1) the number of FTEs, including labor-
related overhead costs such as training and leave, for operating the
Error Resolution System (ERS) and (2) the number of errors corrected by
ERS during fiscal year 2002. According to IRS officials, ERS personnel
are hired at GS-4 and the journeyman level is GS-6. Using the number of
ERS FTEs, the number of errors corrected by ERS staff, and the GS-5,
step 1 salary effective January 2002, as the salary for ERS staff
members, we estimated that IRS incurred $0.36 in costs per error. (IRS
officials were unable to provide the actual costs for ERS staff. We
chose to use GS-5, step 1, because beginning ERS staff are GS-4s and
more experienced ERS staff are GS-6s.) Total error correction costs may
be higher because the $0.36 estimate does not include various other
costs such as nonlabor-related overhead for equipment and supplies.
[B] IRS contracted with a firm to conduct focus groups to test EIC and
Child Tax Credit forms and instructions. The contract was for $54,000
and IRS spent an additional $2,000 on related travel costs. Because one
contractor conducted the tests for both credits, IRS officials said it
would be difficult to allocate the $56,000 between the tests for the
two credits.
[End of table]:
To provide some perspective on how the potential benefits to taxpayers
from testing EIC and Child Tax Credit forms could affect the overall
benefits and costs of testing, we next looked at potential reduced
burden from credit claimants receiving fewer notices due to reduced
errors. First, we assumed that on average all taxpayers receiving an
error notice from IRS take 2 or 5 minutes to deal with the notice.
Based on those assumptions, we calculated the value to taxpayers of the
time saved (using minimum wage levels) from not having to deal with IRS
error notices.[Footnote 17] We used the same assumed reductions in
errors due to testing that we made for table 4 and we netted taxpayers‘
savings with the savings shown in table 4 for IRS alone.
As table 5 shows, including testing-related benefits to taxpayers from
decreased errors suggests that in the first year following testing of
EIC and Child Tax Credit forms and instructions, the net benefit to
taxpayers and IRS combined could have been positive except for our
lowest assumption about the degree to which testing may have reduced
taxpayer errors--our 10 percent assumption.
Table 5: Illustration of Potential Savings to Taxpayers and IRS in 2000
from Error Reductions Alone from Testing EIC and Child Tax Credit Forms
and Instructions:
Form and instruction: EIC; Assumed number of errors eliminated due to
testing: 250,534; Value to taxpayers in time saved (2 minutes per
notice): $43,000; Value to taxpayers in time saved (5 minutes per
notice): $108,000; Potential IRS cost savings assuming $56,000 cost to
perform test: $34,000; Potential net savings to taxpayers and IRS from
error reductions: 2 minutes: $77,000; Potential net savings to
taxpayers and IRS from error reductions: 5 minutes: $142,000.
Assumed number of errors eliminated due to testing: 125,267; Value to
taxpayers in time saved (2 minutes per notice): 22,000; Value to
taxpayers in time saved (5 minutes per notice): 54,000; Potential IRS
cost savings assuming $56,000 cost to perform test: (11,000); Potential
net savings to taxpayers and IRS from error reductions: 2 minutes:
11,000; Potential net savings to taxpayers and IRS from error
reductions: 5 minutes: 43,000.
Assumed number of errors eliminated due to testing: 50,107; Value to
taxpayers in time saved (2 minutes per notice): 9,000; Value to
taxpayers in time saved (5 minutes per notice): 22,000; Potential IRS
cost savings assuming $56,000 cost to perform test: (38,000; Potential
net savings to taxpayers and IRS from error reductions: 2 minutes:
(29,000); Potential net savings to taxpayers and IRS from error
reductions: 5 minutes: (16,000).
Form and instruction: Child tax credit; Assumed number of errors
eliminated due to testing: 248,053; Value to taxpayers in time saved (2
minutes per notice): 43,000; Value to taxpayers in time saved (5
minutes per notice): 106,000; Potential IRS cost savings assuming
$56,000 cost to perform test: 33,000; Potential net savings to
taxpayers and IRS from error reductions: 2 minutes: 76,000; Potential
net savings to taxpayers and IRS from error reductions: 5 minutes:
139,000.
Assumed number of errors eliminated due to testing: 124,026; Value to
taxpayers in time saved (2 minutes per notice): 21,000; Value to
taxpayers in time saved (5 minutes per notice): 53,000; Potential IRS
cost savings assuming $56,000 cost to perform test: (11,000; Potential
net savings to taxpayers and IRS from error reductions: 2 minutes:
10,000; Potential net savings to taxpayers and IRS from error
reductions: 5 minutes: 42,000.
Assumed number of errors eliminated due to testing: 49,611; Value to
taxpayers in time saved (2 minutes per notice): 9,000; Value to
taxpayers in time saved (5 minutes per notice): 21,000; Potential IRS
cost savings assuming $56,000 cost to perform test: (38,000); Potential
net savings to taxpayers and IRS from error reductions: 2 minutes:
(29,000); Potential net savings to taxpayers and IRS from error
reductions: 5 minutes: (17,000).
Source: GAO.
Note: GAO analysis.
[End of table]:
Finally, to illustrate the potential benefits if testing EIC and Child
Tax Credit forms and instructions made them clearer and thereby reduced
taxpayers‘ time needed to understand and complete the credit forms, we
calculated the value of time saved by taxpayers (using minimum wage
levels) in understanding and completing EIC and Child Tax Credit forms
assuming the time saved was 1 minute. Unlike for tables 5 and 6, all
taxpayers who used the form or instructions to determine whether they
qualified for either credit may have saved time if testing contributed
to clearer EIC and Child Tax Credit forms and instructions. However,
because we did not know how many taxpayers might have used the forms
and instructions for this purpose, in calculating the value of time
taxpayers may have saved we used only the number of taxpayers who
claimed these credits and did not use paid preparers to prepare their
tax returns.
Table 6 shows that if testing the credits‘ forms and instructions
helped clarify them and that led taxpayers to take 1 minute less to
understand and complete the forms, credit claimants would have saved a
total of about 240,000 hours worth $1.2 million at minimum wage levels.
Table 6: Illustration of Potential Benefit to Taxpayers in 2000 if
Testing Reduced the Time Needed to Understand and Complete EIC and
Child Tax Credit Forms:
Form and Instruction: EIC; Number of returns claiming the credit
(millions)[A]: 6.2; Total hours saved assuming 1 minute less per
taxpayer: 100,000; Potential value to taxpayers of time saved
(millions): $0.5.
Form and Instruction: Child Tax Credit; Number of returns claiming the
credit (millions)[A]: 8.4; Total hours saved assuming 1 minute less per
taxpayer: 140,000; Potential value to taxpayers of time saved
(millions): 0.7.
Form and Instruction: Totals; Number of returns claiming the credit
(millions)[A]: 14.6; Total hours saved assuming 1 minute less per
taxpayer: 240,000; Potential value to taxpayers of time saved
(millions): 1.2.
Source: GAO.
Note: GAO analysis.
[A] This is the number of tax returns prepared by taxpayers claiming
the credits. Since taxpayers who used a preparer might not need to read
and understand the forms and instructions, we did not include them in
this table. Both the number of EIC and Child Tax Credit returns are
based on the percentage of returns prepared by taxpayers who claimed
EIC since data on the percentage of returns prepared by taxpayers who
claimed the Child Tax Credit were not available.
[End of table]:
To provide another perspective on the potential magnitude of benefits
and costs associated with testing changes to forms and instructions, we
also looked at IRS‘s experience with the rate reduction credit. This
one-time credit was enacted in June 2001.[Footnote 18] When 2001 tax
returns were processed during 2002, over 8 million returns had errors
related to the credit.[Footnote 19] IRS did not test the instructions
for computing the rate reduction credit that was included on the Form
1040 for tax year 2001. According to IRS officials, they did not test
the instructions because the credit was a one-time event, and in their
judgment, they had insufficient time to test it. We reported that some
of the taxpayers‘ errors were probably due to taxpayers not
understanding IRS‘s instructions on how to compute the credit. We also
reported that the demand for telephone assistance related to the credit
was significant during the 2002 filing season, and that some of these
calls, based on Taxpayer Advocate Service information, were made
because taxpayers did not understand how to compute the
credit.[Footnote 20]
Using the same approach to illustrate whether IRS alone may have
realized benefits in excess of its testing costs as we did for EIC and
Child Tax Credit changes, we developed the illustration shown in table
7.[Footnote 21] As shown, considering only IRS‘s cost and assuming that
all errors were corrected by IRS using its math error procedures and
assuming IRS would have spent the same amount to test the rate
reduction credit instructions as it did for EIC and Child Tax Credit
tests, IRS may have been able to save between $233,000 and $666,000.
Although this case is somewhat atypical since the rate reduction credit
affected essentially all individual taxpayers and the number of errors
related to the credit was unusually high, these figures illustrate that
the potential for savings to IRS alone from testing instructions at
times can substantially exceed its testing costs.
Table 7: Illustration of Potential Cost Savings to IRS in 2002 if the
Rate Reduction Credit Instructions Had Been Tested and Errors Were
Reduced:
Number of returns with rate reduction credit errors: 8,025,851;
Differing assumed levels of errors avoided: 25%; Assumed number of
errors eliminated by testing: 2,006,646; Estimated direct labor cost to
correct an error: $0.36; Estimated cost to correct eliminated errors:
$722,000; Potential IRS cost savings assuming $56,000 cost to perform
test: $666,000.
Differing assumed levels of errors avoided: 15%; Assumed number of
errors eliminated by testing: 1,203,878; Estimated direct labor cost to
correct an error: 0.36; Estimated cost to correct eliminated errors:
433,000; Potential IRS cost savings assuming $56,000 cost to perform
test: 377,000.
Differing assumed levels of errors avoided: 10%; Assumed number of
errors eliminated by testing: 802,585; Estimated direct labor cost to
correct an error: 0.36; Estimated cost to correct eliminated errors:
289,000; Potential IRS cost savings assuming $56,000 cost to perform
test: 233,000.
Source: IRS and GAO.
Note: GAO analysis of IRS data.
[End of table]:
However, just as with EIC and Child Tax Credits, taxpayers would have
benefited if testing had been done, led to clearer instructions and
consequently led to fewer taxpayer errors for the rate reduction
credit. Using the same approach we used for EIC and Child Tax Credits,
table 8 shows the potential taxpayers‘ savings from dealing with fewer
rate reduction credit error notices and the net savings to taxpayers
and IRS.
Table 8: Illustration of Potential Savings to Taxpayers and IRS in 2002
from Potential Error Reductions if Testing the Rate Reduction Credit
Had Reduced Taxpayer Errors:
Number of returns with rate reduction credit errors: 8,025,851; Assumed
number of errors eliminated by testing: 2,006,646; Value to taxpayers
of time saved (2 minutes per notice): $344, 000; Value to taxpayers of
time saved (5 minutes per notice): $861,000; Potential IRS cost savings
assuming $56,000 cost to perform test: $666,000; Potential net savings
to taxpayers and IRS from error reductions: 2 minutes: $1,010,000;
Potential net savings to taxpayers and IRS from error reductions: 5
minutes: $1,527,000.
Assumed number of errors eliminated by testing: 1,203,878; Value to
taxpayers of time saved (2 minutes per notice): 207,000; Value to
taxpayers of time saved (5 minutes per notice): 517,000; Potential
IRS cost savings assuming $56,000 cost to perform test: 377,000;
Potential net savings to taxpayers and IRS from error reductions: 2
minutes: 584,000; Potential net savings to taxpayers and IRS from
error reductions: 5 minutes: 894,000.
Assumed number of errors eliminated by testing: Assumed number of
errors eliminated by testing: 802,585; Value to taxpayers of time saved
(2 minutes per notice): Value to taxpayers of time saved (2 minutes per
notice): 138,000; Value to taxpayers of time saved (5 minutes per
notice): Value to taxpayers of time saved (5 minutes per notice):
344,000; Potential IRS cost savings assuming $56,000 cost to perform
test: Potential IRS cost savings assuming $56,000 cost to perform test:
233,000; Potential net savings to taxpayers and IRS from error
reductions: 2 minutes: 2 minutes: 371,000; Potential net savings to
taxpayers and IRS from error reductions: 5 minutes: 5 minutes: 577,000.
Source: GAO.
Note: GAO analysis.
[End of table]:
Table 9 shows that if testing had been done and it improved the clarity
of the instructions enough to save taxpayers, on average, 30 seconds in
understanding whether and how they needed to complete the credit line
on their tax returns, the savings would have been larger than savings
to IRS and taxpayers from avoided errors alone.[Footnote 22]
Table 9: Illustration of Potential Benefit to Taxpayers in 2002 if
Testing Reduced the Time Needed to Understand and Complete the Rate
Reduction Credit:
Number of returns affected by the credit[A]: 50,000,000; Total hours
saved assuming 30 seconds less per taxpayer: 415,000; Potential value
to taxpayers of time saved (millions): $2.1.
Source: GAO.
Note: GAO analysis.
[A] This is the number of paper returns prepared by taxpayers. Since
taxpayers who used a preparer might not need to read and understand the
instructions, we did include them in this table. We only included the
number of paper returns because information was not readily available
on the number of returns filed electronically that were completed by
taxpayers rather than a preparer.
[End of table]
IRS Has Not Addressed Constraints to Increased Testing:
Although IRS officials said that making greater use of testing to
improve clarity of forms and instructions could be beneficial,
officials have not addressed the two constraints--time and resources--
that they state limit their ability to do more testing of changes to
forms and instructions. Time constraints are not binding for some
changes IRS considers to forms and instructions, although IRS cannot
realistically test the unknown portion of the changes that are due to
laws passed shortly before, or even after, the effective dates for the
forms. Also, IRS‘s procedures for developing and revising forms (1) do
not clearly specify which draft version of forms and instructions
should be tested with taxpayers or (2) when in the annual forms
development cycle testing should occur. In addition to tight time
frames, officials also say that limited resources, such as only one
person responsible for coordinating all testing efforts in the Forms
and Publications Division, preclude them from increasing tests of forms
and instructions. However, IRS has not documented which changes to
forms and instructions likely would benefit from testing or
demonstrated the benefits that are gained when testing is done. IRS‘s
planning and budgeting process uses such information in determining the
level of resources to be allocated to various units.
Tight Time Frames Do Not Always Exist and IRS‘s Procedures Do Not
Always Facilitate Testing:
IRS officials told us that when new tax laws are enacted during the
year that require IRS to create or revise tax forms and instructions in
time to distribute them to taxpayers by January 1, the start of the
tax-filing season, they lack time to test the forms and instructions
before distributing them to taxpayers. However, not all changes to
forms and instructions are time constrained and IRS‘s procedures lack a
clear target for which version of forms and instructions should be
tested with taxpayers.
While sufficient data were not available to determine the portion of
changes IRS makes to forms and instructions that cannot be tested due
to time constraints, not all changes are time constrained. Due to the
variability in the time that may be required to test a form or
instruction and in the amount of time IRS needs to develop the initial
form or instruction to be tested, we cannot say definitively when IRS
may or may not have sufficient time to conduct tests. In some cases,
IRS likely could have sufficient time to do testing when it identifies
a needed change to forms or instructions itself since it largely
controls the scheduling of this work. Similarly, when the Congress
passes a law that is not effective until a future tax year, or that
contains provisions that are not effective until a future tax year, IRS
may have sufficient time to conduct tests. For example, the Economic
Growth and Tax Reconciliation Act of 2001 was passed on June 7, 2001,
with some provisions effective for tax year 2001, but others with later
effective dates. The provisions modifying education Individual
Retirement Accounts were effective for taxable years beginning after
December 31, 2001. This gave IRS approximately 16 months to develop and
test any modifications to tax forms and instructions and make final
revisions before those forms and instructions needed to go to printing
for distribution by January 2003. When a law affects the current tax
year, i.e., changes how taxpayers will need to calculate their taxes in
the next tax-filing season, IRS is less likely to have sufficient time
to test. Even in such a case, however, the new law may be passed early
enough to allow testing.
IRS‘s current procedures for developing and revising forms and
instructions do not clearly specify which draft version of forms and
instructions should be tested with taxpayers or when in the annual
forms development cycle testing should occur. Officials said that draft
forms may be tested with taxpayers either before or after they are
posted to IRS‘s website for external comments by the public, tax
practitioners, software developers, and others. Tax Forms and
Publications Division officials said that they consider the particular
circumstances surrounding the development of each form and instruction
when deciding which version of a draft form or instruction they should
test. However, because IRS does not have a clear targeted time for
testing, IRS‘s ability to plan and conduct tests maybe constrained. If
IRS‘s procedures defined a point in the annual forms development cycle
where a version of a draft form or instruction would be available for
testing, IRS would be able to establish processes and deadlines
designed to ensure that the opportunity for testing is realized.
To the extent that a draft version of a form or instruction is
available for testing early in the process, it would give IRS a fuller
range of options for testing. For example, if IRS tested draft versions
of forms and instructions before or during the approximately 3-week
period that the form is available on its Web site, this would minimize
any additional calendar time that testing might otherwise add to IRS‘s
forms development process.
Figure 2 shows the points in IRS‘s annual forms development process
where testing can occur. As illustrated, testing may be conducted early
in the process and late in the process.
Figure 2: Simplified Overview of Where Testing Can Occur in IRS‘s
Annual Tax Forms Development Process:
[See PDF for image]
Note: GAO analysis of IRS data.
[End of figure]:
Testing earlier drafts of forms and instructions would also enable
officials to select from various testing alternatives depending on how
early a draft is available for testing. We did not find a uniform
amount of time needed to test a change to a form or instruction. At the
low end of the spectrum, an official from NCHS said that it takes about
7 weeks to test that agency‘s questionnaires using one-on-one
interviews. IRS officials estimated that when IRS employees are used as
focus group participants it requires about 8 to 12 weeks to schedule
and conduct the tests, analyze the data, and prepare a report
summarizing the results. IRS officials estimated that when they
contract with a private firm to conduct focus groups using private
citizens, 24 to 32 weeks are required to obtain a contract, recruit
participants, conduct the tests, analyze the results, and prepare a
report. This time frame is based on using regular contracting processes
involving developing a statement of work, soliciting bids, and
selecting a contractor. Contract options exist that enable agencies to
identify a firm or group of firms qualified to undertake work so that
an expedited task order procedure can be used to select a firm for when
needs arise. According to IRS officials, they recently entered into a
multiyear contract with two vendors that will enable them to issue task
orders when work is needed.
Data Are Not Collected That Could Be Used by IRS Management to
Determine the Proper Allocation of Resources to Support Testing:
Although IRS‘s Tax Forms and Publications Division officials believe
current resources are insufficient to support more testing of forms and
instructions, they do not have some of the information needed to
determine whether to allocate additional resources. This information is
not available at least in part because division guidelines and policies
do not require that it be gathered.
Officials said that because they have so few staff available to conduct
tests and have a limited budget to contract for testing, they could not
increase the number of tests they perform. According to the officials,
currently only 1 of 103 persons in the division is trained in testing
methods. In addition to other duties, this person coordinates the tests
for the division, such as the test of EIC forms and instructions
completed in 1999 by a private vendor and the test of the innocent
spouse application form completed by IRS in 2002. Officials also told
us some staff who are primarily responsible for creating and revising
tax documents may occasionally assist in conducting tests, such as the
three persons involved in testing the innocent spouse form. Officials
also said the total budget for contract support for the division was
$150,000 in fiscal year 2002, $185,000 in fiscal year 2001, and
$130,000 in fiscal year 2000.
As part of its annual planning and budgeting process, IRS management
determines what resources will be needed to accomplish strategies and
implement programs. IRS‘s planning and budget guidance requires that
each operating unit prepare a business plan that, among other things,
clearly defines priorities and resource requirements. Requests in the
business plan for resources must be substantiated with evidence that
allocating additional resources is justified.
However, the division does not systematically identify when testing
would be beneficial and does not routinely demonstrate the benefits to
taxpayers and IRS that have been gained from such testing. Officials do
not identify which of the many changes it makes to forms and
instructions each year would most likely benefit from testing. Thus,
the officials cannot tell IRS management how many opportunities to
improve forms and instructions may be lost due to current resource
levels. Further, when tests are performed, officials do not identify,
quantitatively or qualitatively, the benefits that taxpayers and IRS
may have realized.
One reason that IRS does not have data on forgone testing opportunities
is that the division lacks formal, written guidelines and procedures
for determining when testing would be beneficial. Currently, testing is
an optional step in the process for developing forms. IRS‘s Tax Forms
and Publications officials said that they decide which forms to test
based on informal guidelines and procedures and input from officials in
IRS‘s four operating division program offices and the Taxpayer Advocate
Service. The informal guidelines and procedures call for officials to
weigh, among other things, whether a form or instruction (a) affects a
large number of taxpayers, (b) has a high error rate based on
taxpayers‘ prior use of the form, (c) is perceived as complex, and (d)
will be used for several filing seasons. Also, according to IRS, the
amount of time available to perform tests is factored into testing
decisions.
These informal guidelines do not require officials to consider in all
cases whether testing would be beneficial and to document the decisions
made. Accordingly, even if the informal guidelines are applied, and
officials judge that some forms or instructions could benefit from
testing but cannot be tested due to scarce resources, those decisions
are not made systematically and documented.
Further, although the factors the guidelines suggest taking into
account appear to have evolved from officials‘ experience and therefore
should be useful, they do not consider some pertinent factors that
could affect the benefits likely to be realized from testing. For
instance, the guidelines suggest taking the number of affected
taxpayers into account but not the likely amount of burden they would
face due to unclear forms or instructions. They also do not clearly
call for officials to consider the costs to test forms and instructions
and the benefits that may accrue throughout IRS, such as in telephone
service centers. In addition, these informal guidelines and procedures
automatically exclude testing forms and instructions that will be used
only one time. Also, according to IRS officials, the time frame between
the passage of new tax laws and when the newly created or revised forms
and instructions must be finalized may preclude some forms and
instructions from being tested. Even if one-time-use forms meet other
testing criteria, such as affecting a large number of taxpayers who may
perceive them as complex, IRS will not consider testing them. As the
rate reduction credit situation discussed earlier illustrates, such
automatic exclusions may not be appropriate in all situations.
IRS officials do not have information on the results achieved when
forms and instructions are tested in part because the division does not
have policies that require such evaluations. When IRS obtained
information on the reduction in error rates following testing of EIC
and Child Tax Credit forms and instructions, the studies did not
include collecting other information on the benefits that may have
resulted for taxpayers and for IRS. For instance, the studies did not
estimate the savings IRS may have realized in its telephone and walk-in
service due to increased form clarity.
Capturing fuller information on the results of testing would be
consistent with IRS‘s strategic planning and budgeting process, which
emphasizes assessing the impact of current programs to efficiently
allocate resources. Further, by evaluating results of testing
decisions, IRS officials would be able to determine if their testing
guidelines and procedures lead to good decisions about when testing is
most likely to be beneficial. They may also be able to see if the
methods they use to test--for example, focus groups formed by IRS
employees or one-on-one interviews with individuals--yield the most
effective test results.
Conclusions:
IRS continually faces the daunting task of developing and revising tax
forms and instructions to administer our ever-changing set of federal
tax laws. Taxpayers rely on IRS for forms and instructions that are as
clear and easy to understand as possible given the complexity of the
tax laws and providing clear materials is a key goal of IRS‘s Tax Forms
and Publications Division.
In attempting to meet this goal, IRS has tested an average of one set
of forms and instructions each year over the last 5 years. In contrast,
officials from three federal agencies that routinely collect
information from the public say that testing documents for clarity
before using them is their standard practice. They do so because they
believe testing will ensure that their data collection documents are
clear and that individuals will understand them and complete them
accurately.
Although it is difficult to gauge how much testing alone contributes to
the clarity of tax forms and instructions, IRS officials believe
testing has contributed to significant declines in taxpayer errors.
Illustrations we developed based on IRS‘s experience in testing forms
and instructions suggest that IRS can completely recover its testing
costs in the first year following testing in some circumstances and
that when savings to taxpayers from more understandable forms and
instructions are considered, total benefits even in the first year
following tests can be several times IRS‘s testing costs.
Although they recognize that testing is beneficial, officials say time
constraints and limited resources preclude more testing. However, IRS‘s
procedures do not clearly specify when draft versions of forms and
instructions should be available for testing. Having a clearly defined
point where testing would be performed would facilitate establishing
procedures and deadlines to better ensure that testing could be done
even within IRS‘s annual forms update cycle. Further, IRS officials do
not have information that would help IRS management to determine
whether to allocate additional resources to support enhanced testing.
Because IRS lacks standard written procedures for testing, officials
have not documented cases where testing would likely be beneficial and
have not demonstrated the benefits that are gained from testing.
Recommendations for Executive Action:
Because testing could potentially yield clearer and more understandable
tax forms and instructions, thereby producing benefits both to
taxpayers and IRS, we recommend that the Acting Commissioner of
Internal Revenue take the following actions.
* Develop written criteria for determining which changes to tax forms
and instructions should be tested with taxpayers before publication.
:
* Develop official written guidance that incorporates those criteria
and ensure that the guidance requires staff that develop new or revised
forms and instructions to document which changes would merit testing
and why.
:
* Clarify procedures by designating when in the annual forms
development process that a draft version of forms and instructions
should be available for testing with taxpayers.
:
* Ensure that an appropriate range of evaluations are conducted of
tests that are performed to better establish the costs and benefits of
performing tests and to refine IRS‘s approach to testing on the basis
of lessons learned.
:
* Use information gained from documenting when changes to forms or
instructions likely would be beneficial and from evaluations of tests
to reassess an appropriate level of resources to perform testing.
Agency Comments and Our Evaluation:
The Acting Commissioner of Internal Revenue provided written comments
on a draft of this report in an April 7, 2003, letter, which is
reprinted in appendix III. The Acting Commissioner agreed with our
recommendations. We are encouraged that IRS plans to implement all but
one of our recommendations in time for the 2004 forms development
cycle. Understandably, the remaining recommendation to ensure that an
appropriate range of evaluations is conducted of tests would take more
time to put into practice. The Acting Commissioner also provided
additional comments and observations on our draft report.
The Acting Commissioner commented that the crux of our report is that
we do not believe IRS has performed adequate testing on new and revised
tax forms and instructions due to a lack of resources. He said that
resources for testing forms and instructions have been adequate for the
testing IRS wanted to perform. While not questioning whether resources
were adequate for the testing IRS performed, we concluded that IRS
officials do not have information needed to determine the level of
resources that should be allocated to testing forms and instructions.
Accordingly, we recommended that IRS systematically identify
opportunities to improve forms and instructions through testing and to
evaluate the costs and benefits when testing is done.
Although agreeing that testing is beneficial, the Acting Commissioner
also said that there are significant staff costs associated with
testing that are not included in our cost analysis. We recognize that
our analysis excluded staff costs and as stated in our draft report we
sought to be conservative in estimating the benefits of testing, in
part because we did not have information on the full range of costs IRS
incurs when undertaking projects to tests forms and instructions.
During the course of our work, we requested estimates of staff costs
for testing but IRS was unable to provide them. Nevertheless, at least
in the cases we illustrated the potential benefits of testing were so
much greater than the costs that including staff costs likely would not
have substantially changed the results of our illustrations.
The Acting Commissioner expressed concern about whether IRS could have
forms and instructions ready for the filing season if testing was done
late in the forms development cycle as shown in our figure 2 depicting
IRS‘s process. We agree with the Acting Commissioner‘s concern;
however, our figure shows the various points at which testing can occur
in IRS‘s current processes based on interviews with IRS officials and
the documentation they provided us. As IRS implements our
recommendation to clarify when testing should be done, selecting a
point as early as possible would help maximize the number of changes
that can be tested during the annual forms update cycle.
The Acting Commissioner also said that he disagreed with our conclusion
that IRS‘s experience with obtaining feedback on its products is
limited or recent. He said IRS uses various methods to obtain customer
feedback. We agree that IRS uses methods other than testing to obtain
feedback on its forms and instructions. However, our report describes
the potential benefits and costs of testing as a feedback method. In
terms of testing, IRS has only tested five forms and instructions
during July 1997 through June 2002; in our view, this is a limited
number of tests that were conducted during the recent past.
The Acting Commissioner also disagreed that testing would result in
reduced demand for walk-in and toll-free assistance. He said that IRS
lacks data to support such a conclusion and, based on its experience,
new forms generate requests for assistance and error rates on them tend
to be higher. We recognize that there will always be a demand for
taxpayer customer assistance. However, we believe that reduced demand
for assistance is a potential benefit of testing. We note, for example,
that IRS officials seek input from telephone assistors when deciding
which forms or instructions need to be clarified, apparently believing
that clarifying the forms and instructions may help reduce calls to
assistors. Finally, testing is one means for ensuring that even for new
forms and instructions requests for assistance and errors made by
taxpayers will be minimized.
We are sending copies of this report to the Chairman and Ranking
Minority Member of the House Committee on Ways and Means and its
Subcommittee on Oversight; the Secretary of the Treasury; the Acting
Commissioner of Internal Revenue; the Director of the Office of
Management and Budget; and other interested parties. We will make
copies available to others on request. In addition, the report will be
available at no charge on the GAO Web site at http://www.gao.gov.
This report was prepared under the direction of Charlie Daniel,
Assistant Director. If you have any questions regarding this report,
please contact him or me at (202) 512-9110. Key contributors to this
report were David Alexander, Christopher Currie, Ronald La Due Lake,
Anne Laffoon, Veronica Mayhand, Edward Nannenhorn, and Shellee Soliday.
Michael Brostek:
Director, Tax Issues:
Signed by Michael Brostek:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
To determine how often IRS has used taxpayers to test the clarity of
new or revised individual income tax forms and instructions, we
interviewed officials in its W&I Division‘s Tax Forms and Publications
Division in Washington, D.C. We sought to obtain (a) an understanding
of the process IRS used to develop and revise individual income tax
forms and instructions and (b) gather information on the forms and
instructions IRS tested using taxpayers for the 5-year period between
July 1997 and June 2002. We requested information for a 5-year span to
help ensure that the information collected would reflect the amount of
testing usually done by IRS. Our work did not include assessing IRS‘s
processes for developing and revising notices or publications or
assessing the clarity of any specific tax forms or instructions.
To obtain insights on the benefits of testing written documents, we
interviewed officials from three federal agencies in the metropolitan
Washington, D.C., area that perform extensive research and data
collection using private citizens. We contacted NCHS, BLS, and the
Census Bureau because they have broad experience in conducting tests of
the clarity of written documents, such as forms, surveys, and
instructions. We also contacted a private research firm that
specializes in testing forms and surveys for a variety of public and
private sector clients. We also reviewed our own guidance for
developing and using questionnaires.[Footnote 23]3:
To determine the benefits to taxpayers and IRS of testing tax forms and
instructions with taxpayers prior to their use by the public, we took
several steps. First, we interviewed IRS officials including Tax Forms
and Publications Division officials to obtain information on whether
they perceived testing as being beneficial to taxpayers and IRS. We
also interviewed IRS‘s W&I Research Division officials in Indianapolis
to obtain their views on whether testing tax forms and instructions
benefits taxpayers and IRS. The research division officials, among
other things, collect information on taxpayer errors that the Tax Forms
and Publications Division uses when deciding which forms and
instructions to test. These officials provided us with data on changes
in error rates for EIC and Child Tax Credit forms and instructions
before and after IRS revised and tested them.
Second, we developed illustrations of the potential benefits and costs
of IRS‘s testing EIC and Child Tax Credit forms and instructions by
analyzing the data on the changes in error rates obtained from IRS
research officials. To construct our illustrations, we used readily
available data and made certain assumptions. Data were not available on
many of the potential benefits of the changes and on the full range of
costs IRS incurred to conduct the tests. We developed similar
illustrations of potential benefits to taxpayers and IRS if the rate
reduction credit instructions had been tested. In all cases, the
illustrations we developed are not actual assessments of the costs and
benefits that were associated with testing forms and instructions, or
that would have resulted if testing had occurred. In developing the
illustrations we sought to be conservative in estimating benefits, in
part because we did not have information on the full range of costs IRS
incurred to test forms and instructions. Because we had to make various
assumptions, our illustrations undoubtedly vary from actual costs and
benefits.
To determine the potential cost savings to IRS of testing these forms
and instructions, we first estimated IRS‘s costs to correct a taxpayer
error. Our estimates of IRS‘s cost to correct errors were limited to
the labor cost associated with correcting errors in IRS‘s math error
program. IRS provided us the number of FTE staff years for operating
the ERS that was used to detect and correct math errors and the number
of errors corrected by ERS during fiscal year 2002. We used this
information to calculate an average labor cost to correct math errors
that we then applied to reductions in EIC and Child Tax Credit errors
as well as to error reductions that might have resulted from testing
the rate reduction credit. Total error correction costs may be higher
because we did not include a number of other costs associated with
correcting errors. For example, we excluded printing and postage costs
for the notices sent to taxpayers. IRS notices often cover more than
one issue associated with a tax return and data were not readily
available to determine what portion of the postage cost might be
attributable to EIC or Child Tax Credit issues alone. In addition, the
estimate does not include costs such as equipment and rent. We did not
specifically test the accuracy of the cost information provided;
however, our audits of IRS‘s annual financial statements have raised
concerns regarding IRS‘s ability to identify all costs associated with
a given program or activity.[Footnote 24]4:
Because data were not available on the extent that testing reduced
errors related to EIC and Child Tax Credit forms and instructions, we
assumed different percentage reduction rates in errors due to testing
these forms and instructions. Our illustrations show potential savings
based on assumed percentage reduction in errors attributable to testing
of 50, 25, and 10 percent. Using our estimate of IRS‘s costs to correct
an error and the assumed number of errors eliminated by testing, we
arrived at the costs that would have been incurred to correct those
errors. The difference between costs to correct eliminated errors and
the costs to test is the potential cost savings from testing. Cost
savings due to reduced errors likely would not mean reductions to IRS‘s
budget. Further, savings likely would mean that IRS would provide
services to other taxpayers or would pursue other compliance or tax
collection activities that it would otherwise have been unable to do.
To determine IRS‘s costs to test forms and instructions, we used IRS‘s
actual contract costs for testing EIC and Child Tax Credits in 1999.
IRS spent a total of about $56,000 to test both EIC and Child Tax
Credit forms and instructions with focus groups. Although the cost of
contracted support for testing each form and instruction individually
likely would have been somewhat lower than this, we applied the total
cost in each case.[Footnote 25]5 Because IRS could not provide data, we
did not include the costs associated with IRS letting and managing the
contract or the cost for Tax Forms and Publications Division staff to
work with the contractor in conducting and managing the tests or the
cost of any other IRS staff that were involved in these tests.
To determine the potential benefits to taxpayers from testing EIC and
Child Tax Credit forms and instructions, we calculated estimated values
to taxpayers of the time saved if testing improved form and instruction
clarity, thereby reducing taxpayer errors and the burden of dealing
with IRS error notices. We also estimated taxpayers‘ time saved if
testing reduced the time needed to understand and complete tax forms
and instructions. We developed similar illustrations for potential
benefits to taxpayers if the rate reduction credit had been tested. Our
benefit illustrations were based on a series of assumptions. For
example, to estimate the value to taxpayers of time saved from not
having to deal with an IRS notice, we applied the minimum wage rate
that was in effect after IRS revised and tested the forms and
instructions to estimates of time taxpayers might save by not having to
deal with an IRS notice.[Footnote 26]6 For these illustrations we
assumed, on average, that taxpayers who received an IRS error notice
might spend either 2 or 5 minutes to deal with it. Data were
unavailable on how much time taxpayers actually spend dealing with
IRS‘s notices; however, according to Taxpayer Advocate Service
information, IRS‘s notices are difficult for taxpayers to understand.
Further, taxpayers who decide to contest an IRS notice may take time to
call or write letters to IRS or to contact and work with a tax
preparer. Data were not readily available to determine what portion of
taxpayers who received an EIC or Child Tax Credit error notice
contested IRS‘s change to their tax returns.
For our illustrations of the estimated value of time saved by taxpayers
if testing reduced the time needed to understand and complete EIC and
Child Tax Credit forms and instructions, we assumed that taxpayers
would save on average 1 minute by using clearer forms and instructions.
We excluded from our illustrations those tax returns prepared by
preparers because the taxpayers might not have had to read and
understand the forms and instructions. Because data were not available
on the number of tax returns claiming the Child Tax Credit that were
prepared by paid preparers, we reduced the number of returns claiming
the credit by the same percentage of EIC returns that were prepared by
preparers. The percentage of EIC claimants using paid preparers exceeds
the average for all taxpayers. For the rate reduction credit, our
illustration is based on a 30-second time savings and the number of
taxpayers who filed on paper and did not use preparers. We chose a 30-
second potential savings for this illustration because many taxpayers
would have had to read only part of the instructions to determine what
to do. We aggregated the times for all taxpayers and multiplied the
total hours saved by the prevailing minimum wage rate to arrive at
estimated benefits to taxpayers.
To determine whether any factors limited IRS‘s ability to use
individual taxpayers to test forms and instructions and, if so, how
these factors can be addressed, we interviewed IRS‘s Tax Forms and
Publications Division‘s officials and analyzed supporting data they
provided us. Regarding officials‘ view that they lacked sufficient time
to do more testing, we reviewed information on the amount of time IRS,
NCHS, and the private research firm we contacted took to perform
various types of tests. We also reviewed IRS‘s process for developing
new and revised forms and instructions and determined how many weeks
were available between the dates that various laws were enacted or
their provisions became effective and IRS‘s normal October 1st deadline
for printing. Finally, regarding officials‘ view that they lacked
sufficient resources to do more testing, we obtained information on the
resources available within IRS for testing.
We performed our work from May 2002 through March 2003 in accordance
with generally accepted government auditing standards.
[End of section]
Appendix II: Use of Testing in NCHS, Census, and BLS:
Like IRS, the three federal agencies we contacted create written
documents to be completed by the public. The agencies create documents
such as forms, surveys, and questionnaires that they use to collect
information from the public to fulfill their missions. Unlike IRS,
described below, these agencies routinely test their forms, surveys,
and questionnaires prior to distribution to the public.
* The National Center for Health Statistics, the nation‘s principal
health statistics agency, compiles statistical information to guide
actions and policies relevant to public health and health policy.
According to NCHS, obtaining accurate and usable health information is
crucial to successfully fulfilling its mission to provide reliable
information to the Centers for Disease Control and Prevention and the
Department of Health and Human Services. NCHS collects information
through various sources including questionnaires that it develops and
administers. Researchers at NCHS told us that in support of their
research they administer surveys and questionnaires each year in
addition to developing questionnaires used by the Centers for Disease
Control and Prevention. They test each questionnaire using one-on-one
interviews. When documents pertain to a particular rather than a
general population, the researchers recruit participants with
characteristics similar to those persons who might be completing the
forms or questionnaires. For example, researchers recruited asthmatics
to test a questionnaire related to asthma. NCHS tests forms in one-on-
one settings in which a participant may be asked to work through a form
while a moderator observes and then later interviews the participant.
This approach allows the researcher to identify specific points at
which the forms were confusing or problematic and learn why the
participant had difficulty. NCHS prefers to use one-on-one interviews
when conducting tests because this method closely resembles the ways in
which individuals will be completing the documents since individuals
will likely complete the documents by themselves.
:
* The Census Bureau is the principal agency responsible for collecting
and providing data about the people and the economy of the United
States. An accurate census is important because census results are used
to reapportion seats in the House of Representatives, redraw
congressional districts and other political boundaries, and address
countless other public and private data needs. The Census Bureau
collects information through short-form and long-form questionnaires
that it develops, tests, and administers. In preparation for the 2000
Census, the Congress budgeted millions of dollars to develop and test
questionnaires during the 1990s. The Census Bureau‘s policy requires
that demographic survey questionnaires be tested. It has used focus
groups and one-on-one interviews to test its questionnaires and forms.
For example, in fiscal year 1996, the Census Bureau decided to make
fundamental changes to the traditional census design such as shortening
census questionnaires. In that year, it budgeted funds to test, among
other things, respondents‘ understanding of race and ethnicity
questions. The Census Bureau has also conducted detailed cost-benefit
analyses of alternative designs; in 1992 it tested the simplified
questionnaire in order to gauge whether the new form would increase
response rates and reduce costly follow up with households that did not
respond to the census.[Footnote 27]7
:
* The Bureau of Labor Statistics, the principal fact-finding agency for
the federal government in the broad field of labor economics and
statistics, also depends on clear and understandable written documents
to collect accurate information from the public. According to BLS‘s
policy, testing documents such as forms and surveys prior to use by the
general public should be undertaken to help identify factors that may
impede users‘ ability to understand forms or surveys. Then these
factors can be addressed in order to improve the clarity of written
documents and increase the accuracy of responses. Testing should be
done in the early stages of document development so that any problems
with clarity can be identified early. BLS routinely tests its written
documents using focus groups and one-on-one interviews, and uses the
results of the tests to make improvements to the documents.
:
[End of section]
Appendix III: Comments from the Internal Revenue Service:
DEPARTMENT OF THE TREASURY INTERNAL REVENUE SERVICE WASHINGTON, D.C.
20224:
COMMISSIONER:
April 7, 2003:
Mr. Michael Brostek Director, Strategic Issues United States General
Accounting Office 441 G Street, N.W.
Washington, D.C. 20548:
Dear Mr. Brostek:
We appreciate the opportunity to review and provide comments on the
draft report titled ’Tax Administration: IRS Should Reassess the Level
of Resources for Testing Forms and Instructions (GAO-03-486).“ We value
the input of the General Accounting Office on our tax forms and
publication development process. As you know, providing clear and
understandable tax forms and instructions is a goal the IRS has always
pursued to help simplify taxpayers‘ tax-related obligations and reduce
taxpayer burden in complying with the tax laws. Along with the general
comments we provided on the report to your staff, we would like to
offer the following observations:
* We appreciate that changes were made to the draft report as a result
of preliminary comments we provided. The crux of the report is that GAO
does not believe IRS has performed adequate testing on new and revised
tax forms and instructions due to lack of resources. We would like for
the record to reflect the resources for testing forms and instructions
have been adequate for the testing we wanted to do. However, as stated
in our response to the recommendations, we agree to establish a means
of documenting how decisions are made about testing. We also agree that
developing a cost-benefit analysis to assess the results of testing
will help address this issue.
* We fully agree it is beneficial to test certain new or redesigned
products. However, there are significant costs in staff resources
(including training, travel, salaries, etc.) for creating or
redesigning the tested product that are not included by GAO in the cost
analysis. In the future, we will assess possible cost analysis methods
that take all of the associated factors into account when designing new
products.
We agree with GAO that other issues outside of our control, such as a
taxpayer‘s poor math skills, will continue to cause errors. These all
contribute to the difficulty in developing a cost-benefit analysis on
the value of any testing.
* We currently use a number of sources to identify materials to be
tested. In conjunction with Wage and Investment Research Group 3, Tax
Forms and Publications can identify the products on which the greatest
number of errors occur and those impacting the largest number of
taxpayers. These criteria are of great value in selecting materials to
be redesigned and tested.
There are situations when late legislation severely limits our time to
develop materials for public use. The rate reduction credit in 2001 is
a prime example. However, these situations are relatively uncommon.
* We recognize the benefits of obtaining feedback from customers on our
products and strive to obtain feedback through a variety of methods in
addition to conducting focus groups. We agree that we conducted five
tests of forms and instructions in focus groups, and one-on-one
interviews between July 1997 and June 2002. We also obtain customer
feedback on our products (forms, instructions, and publications) in
other ways as well. We conduct town hall meetings with the public,
participate in annual Tax Forums, solicit e-mail suggestions from
taxpayers and IRS employees, and regularly meet with professional
groups such as the American Payroll Association, and the American
Institute of Certified Public Accountants. We use feedback from these
sources to make many changes to improve the clarity and
understandability of our products. Our use of these tools to obtain
customer feedback dates back to the late 1970‘s. Therefore, we would
like to go on record as disagreeing with GAO‘s conclusion that IRS‘
experience with obtaining feedback on its products is limited or
recent.
The report reflects GAO‘s conclusion that testing a product will result
in reduced demand for IRS‘ walk-in or toll-free telephone assistance.
Since we currently do not have the data to support such a conclusion,
we disagree with it. While testing a product may indicate where
problems may occur, it cannot predict the number of contacts that will
actually result. Our experience has shown that a new product is likely
to generate requests for assistance because taxpayers are unfamiliar
with it. Similarly, error rates on new forms tend to be higher the
first year a product is issued.
We still are concerned with the box in Figure 2 for considering a test
as changes are made to finalize a form. The amount of time in that
stage of the development process is quite limited. We feel it would not
be wise to conduct testing at the ’Make changes and finalize form“
stage of development. It is too late in the forms development process
to test the form, review the suggested changes, and then incorporate
them in the final product. The printing and distribution of the form in
time for the filing season would be compromised.
My responses to your recommendations for Executive Action are enclosed.
If you need any additional information, please let me know or contact
Floyd Williams, Director, Office of Legislative Affairs at 202-622-
3720.
Sincerely,
Bob Wenzel:
Acting Commissioner:
Signed by
Bob Wenzel:
Enclosure:
Enclosure:
Recommendation:
Develop written criteria to be used in determining which changes to tax
forms and instructions should be tested with taxpayers before
publication.
Response:
We agree with the recommendation and will develop written criteria for
determining which changes to tax forms and instructions should be
tested. These written criteria will be in place for the 2004
development cycle.
Recommendation:
Develop official written guidance that incorporates those criteria and
ensure that the guidance requires staff that develop new or revised
forms and instructions to document which changes would merit testing
and why.
Response:
We agree with the recommendation and will provide official written
guidance for using criteria when developing new and revised forms and
instructions so that documentation is available for decisions made. The
written guidance will be in place for the 2004 development cycle.
Recommendation:
Clarify procedures by designating when in the annual forms development
process that a draft version of forms and instructions should be
available for testing with taxpayers.
Response:
We agree with the recommendation and are revising our operating
procedures to designate when in the annual forms development process a
draft version of forms and instructions should be available for
testing. The revised operating procedures will be in place for the 2004
development cycle.
Recommendation:
Ensure that an appropriate range of evaluations are conducted of tests
that are performed to better establish the costs and benefits of
performing tests and to refine IRS‘ approach to testing on the basis of
lessons learned.
Response:
We agree with the recommendation and will proceed with plans to
establish a means for cost-benefit analyses to be performed to assess,
to the extent possible, the benefits from testing tax forms and
instructions. This will require a major project to assess the reduction
in errors and the impact on processing, customer assistance, compliance
activities, etc., to isolate the benefits from testing tax products
from other assessment:
tools that create benefits (e.g., practitioner feedback). The scope of
the recommendation will most likely require a phased-in approach to
implement the processes that ultimately are agreed to by all affected
organizations within the IRS. We anticipate that analyses can begin
during 2004.
Recommendation:
Use information gained from documenting when changes to forms or
instructions likely would be beneficial and from evaluations of tests
that are performed to reassess an appropriate level of resources to
perform testing.
Response:
We agree with the recommendation. Once the written criteria, official
written guidance, and revised operating procedures are implemented, we
will use the information gathered from testing to reassess the level of
resources available to perform testing. We plan to implement this
recommendation starting in 2004.
[End of section]
FOOTNOTES
[1] Tax forms and instructions as referred to in this report also
include related schedules and worksheets.
[2] 1993 IRS Customer Satisfaction Survey: Public Perceptions of IRS
Service Quality, Contract number TIR 90-0002 (Aug. 27,1993).
[3] U.S. General Accounting Office, Tax Administration: Advance Tax
Refund Program Was a Major Accomplishment, but Not Problem Free, GAO-
02-827 (Washington, D.C.: Aug. 2, 2002).
[4] The Process of Developing Tax Forms for Individual Taxpayers Should
be
Further Improved, TIGTA 2000-40-060 (March 2000); U.S. General
Accounting Office, Tax Administration: IRS Efforts to Improve Forms and
Publications, GAO/GGD-95-34 (Washington, D.C.: Dec. 7, 1994).
[5] Because our review was limited to forms and instructions used by
individual taxpayers, table 1 includes only tests IRS did on these
documents during this period. IRS conducted some other tests during
this period. For example, IRS used focus groups to test 5 publications.
Of the 5 publications, 3 related to individual taxpayers, 1 to business
taxpayers, and one related to both individual and business taxpayers.
IRS estimates that it will revise over 90 publications during 2003.
[6] U.S. General Accounting Office, Developing and Using
Questionnaires, GAO/PEMD-10.1.7 (Washington, D.C.: October 1993).
[7] Floyd J Fowler, 2002; ’Survey Research Methods“; Third Edition;
Sage Publications, Applied Social Research Methods Series, volume 1
(2002). Also see R. Tourangeau, L.J. Rips, and K. Rasinski, The
Psychology of Survey Response (2000).
[8] As noted in table 1, IRS tested five forms during July 1997 through
June 2002. IRS did not determine the effect of testing on error rates
after testing the EIC Schedule in 1998 according to IRS Research
officials. IRS tested Schedule D in 2001. Any change in error rates
would be based on the 2002 filing season, and IRS has not yet analyzed
2002 figures. IRS tested the Innocent Spouse form in 2002 and any
change in error rates will not occur until the 2003 filing season.
[9] Most taxpayers eligible for the EIC have a qualifying child.
[10] Generally, because IRS has more demand for assistance from
taxpayers than it can handle and identifies more potentially
noncompliant tax returns than it can address, benefits from testing are
unlikely to include reductions in IRS‘s overall budget.
[11] In the math error program, IRS uses computer comparisons and
calculations to correct clerical and mathematical errors on tax
returns. These corrections involve addition and subtraction errors;
incorrect social security numbers, filing status and exemptions; and
missing schedules or forms.
[12] Although in many cases taxpayers would not need to respond to a
math error notice, in some cases, such as when IRS has recalculated EIC
or other benefits due to its detection of an apparently invalid social
security number, the taxpayer may be able to provide IRS additional
information to substantiate the original return.
[13] Error correction costs relate to the error resolution process.
When a taxpayer makes an error, in some cases, IRS will correct the
error. For example, if a taxpayer transposes information from the
Schedule D to an incorrect line on the form 1040, IRS can correct this
mistake for the taxpayer. IRS cannot correct certain errors made by
taxpayers such as claiming EIC for a child whose social security number
on the tax return does not match Social Security Administration
records. In this case, because it cannot determine the correct social
security number for the child, IRS will disallow the claim and send an
error notice to the taxpayer that explains what IRS did and what the
taxpayer can do if he/she disagrees with IRS‘s actions.
[14] Department of the Treasury, Internal Revenue Service: Compliance
Estimates for Earned Income Tax Credit Claimed on 1999 Returns
(Washington, D.C.: Feb. 28, 2002).
[15] An FTE consists of one or more employees who collectively work 1
year. For example, one full-time employee or two half-time employees
equal one FTE.
[16] Because IRS officials attributed reduced errors both to their new
approach to structuring EIC guidance and to improvements to the new
approach that resulted from testing, we chose to illustrate reductions
due to testing alone that accounted for half or less of the decline in
errors.
[17] We used minimum wage to value taxpayers‘ time to be conservative
in estimating potential benefits because EIC is targeted to low-income
taxpayers. Minimum wage is a conservative estimate of taxpayers‘ time
even for EIC in that some taxpayers can qualify for EIC at incomes that
are three times minimum wage. For purposes of the Paperwork Reduction
Act, OMB values time for dealing with paperwork requirements like tax
forms at $30 per hour.
[18] Taxpayers who were eligible to receive an advance tax refund in
2001 but who (1) did not receive a check because IRS did not have their
current addresses or (2) did not have enough taxable income in 2000 to
qualify for the maximum amount allowable, may have been entitled to a
rate reduction credit when filing their tax year 2001 returns. In
addition, taxpayers who were not eligible for an advance tax refund,
such as those who did not have taxable income in 2000, may have been
entitled to a rate reduction credit provided they had taxable income in
2001.
[19] This total only includes rate reduction credit errors corrected by
IRS under its math error authority as of September 27, 2002. Some
electronically filed tax returns that IRS rejected during screening for
processing also had rate reduction credit errors.
[20] GAO-02-827.
[21] Consistent with our effort to be conservative in estimating
testing benefits, we assumed that testing would have reduced errors by
no more than 25 percent because many taxpayers only needed to review a
sentence or two of the instructions and did not need to do any
calculations.
[22] We assumed an average taxpayer time savings of 30 seconds because
many taxpayers only needed to review a sentence or two of the
instructions and did not need to do any calculations.
[23] [3] GAO/PEMD-10.1.7.
[24] [4] U.S. General Accounting Office, Major Management Challenges
and Program Risks: Department of the Treasury, GAO-03-109 (Washington,
D.C.: January 2003).
[25] [5] IRS contracted with a firm to conduct focus groups to test EIC
and Child Tax Credit forms and instructions. The contract was for
$54,000 and IRS spent an additional $2,000 on related travel costs.
Because one contractor conducted both tests, IRS officials said it
would be difficult to allocate the $56,000 between the two credits.
[26] [6] Our estimates of the value to taxpayers of time saved from
using clearer forms and instructions are conservative estimates. The
minimum wage rate of $5.15 per hour that we used in our illustrations
is substantially less than OMB‘s estimate of $30 per hour as the
average value of time for individuals and entities to provide
information to the government. Since EIC claimants are lower-income
taxpayers, we used the minimum wage rate. However, some taxpayers would
qualify for EIC even at three times the income of a full time minimum
wage level employee.
[27] [7] U.S. General Accounting Office, Census Reform: Questionnaire
Test Shows Simplification Holds Promise, GAO/T-GGD-92-59 (Washington,
D.C.: July 1, 1992) and U.S. General Accounting Office, Decennial
Census: 1990 Results Show Need for Fundamental Reform, GAO/GGD-92-94
(Washington, D.C.: June 9, 1992) address recommendations that the
Census Bureau simplify its questionnaire in order to improve its
response rate and thereby reduce the number of follow-ups with
nonrespondents and costs associated with follow-up activities.
GAO‘s Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO‘s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO‘s Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ’Today‘s Reports,“ on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select ’Subscribe to daily E-mail alert for newly
released products“ under the GAO Reports heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
