Taxpayer Account Strategy
IRS Should Finish Defining Benefits and Improve Cost Estimates Gao ID: GAO-11-168 March 24, 2011In August 2008, the Internal Revenue Service (IRS) began defining a new strategy for modernizing the way it manages individual taxpayer accounts. The strategy, known as Customer Account Data Engine (CADE) 2, is expected to provide service, compliance, and other benefits to IRS and to taxpayers beginning in 2012. IRS expects to implement CADE 2 in three phases. The first phase is expected to be delivered in 2012, the second in 2014, and the third at a later yet to be determined date. GAO was asked to (1) determine whether IRS has identified the expected benefits of CADE 2 and set targets for measuring success, (2) examine the estimated costs and assess IRS's process for developing them, and (3) assess IRS's process for managing the risks associated with CADE 2 and describe the risks IRS has identified using this process. To do so, GAO reviewed relevant documentation, attended program review meetings, and interviewed IRS officials.
IRS has identified 20 service, compliance, and other benefits for the first phase of CADE 2, including increasing the percentage of refunds processed daily and reducing the number of erroneous notices due to better account information, and has set quantitative targets for most of these benefits. GAO has previously reported that quantitative targets can be useful for tracking program performance. While it may not always be possible to quantify targets, doing so helps to objectively measure the extent to which expected benefits have been realized. However, IRS has not yet finalized expected benefits for the second phase or set related quantitative targets, because, according to officials, these are contingent upon yet to be made design and funding decisions. Nevertheless, addressing the second phase's benefits and related targets as the design is being considered could influence design decisions and help identify early on how systems and processes might be affected. IRS reported preliminary life cycle cost estimates for the first two phases of the CADE 2 program of about $1.3 billion through 2024. This includes about $377 million for development and $922 million for operations and maintenance. IRS's process for developing the preliminary estimates was generally consistent with best practices. However, the agency did not follow three practices intended to improve the credibility of cost estimates. Specifically, IRS did not (1) consistently document excluded costs or provide a rationale for excluding them; (2) use inflation in calculating costs; and (3) perform an analysis to examine the effects of changing ground rules and assumptions. While IRS stated it would perform the analysis of changing ground rules and assumptions in revised estimates to be available by the completion of our audit, until the agency implements all these practices its estimates may not be credible. IRS's process for managing the risks associated with CADE 2 is generally consistent with best practices. Through its process, IRS identified significant risks facing CADE 2, including that filing season and other top information technology investment priorities may result in contention for key resources, the delivery of the first phase of CADE 2 may be delayed if deficiencies identified in requirements are not corrected in a timely manner, and the risk that technical challenges and other risks to implementing the database identified as a result of prototyping efforts may not be addressed. To its credit, IRS has developed mitigation strategies for each identified risk. While IRS is working to ensure CADE 2 is successfully managed, the schedule for delivering the initial phase is nevertheless ambitious. IRS officials have acknowledged this and are taking actions to increase their chances of meeting it, including moving certain activities up, performing others concurrently, and adding checkpoints to monitor the program's status. While these actions may increase the likelihood of meeting the schedule, some of them, such as performing activities concurrently, could potentially introduce more risk to CADE 2's successful development and implementation. GAO's recommendations include (1) identifying all of the second phase benefits, setting the related targets, and identifying how systems and business processes might be affected; and (2) improving the credibility of revised cost estimates by including all costs or providing a rationale for excluded costs, and adjusting costs for inflation. In its comments on a draft of this report, IRS agreed with GAO's recommendations.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: David A. Powner Team: Government Accountability Office: Information Technology Phone: (202) 512-9286GAO-11-168, Taxpayer Account Strategy: IRS Should Finish Defining Benefits and Improve Cost Estimates
This is the accessible text file for GAO report number GAO-11-168
entitled 'Taxpayer Account Strategy: IRS Should Finish Defining
Benefits and Improve Cost Estimates' which was released on March 24,
2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to the Ranking Member, Subcommittee on Financial Services and
General Government, Committee on Appropriations, House of
Representatives:
March 2011:
Taxpayer Account Strategy:
IRS Should Finish Defining Benefits and Improve Cost Estimates:
GAO-11-168:
GAO Highlights:
Highlights of GAO-11-168, a report to the Ranking Member, Subcommittee
on Financial Services and General Government, Committee on
Appropriations, House of Representatives.
Why GAO Did This Study:
In August 2008, the Internal Revenue Service (IRS) began defining a
new strategy for modernizing the way it manages individual taxpayer
accounts. The strategy, known as Customer Account Data Engine (CADE)
2, is expected to provide service, compliance, and other benefits to
IRS and to taxpayers beginning in 2012. IRS expects to implement CADE
2 in three phases. The first phase is expected to be delivered in
2012, the second in 2014, and the third at a later yet to be
determined date.
GAO was asked to (1) determine whether IRS has identified the expected
benefits of CADE 2 and set targets for measuring success, (2) examine
the estimated costs and assess IRS‘s process for developing them, and
(3) assess IRS‘s process for managing the risks associated with CADE 2
and describe the risks IRS has identified using this process. To do
so, GAO reviewed relevant documentation, attended program review
meetings, and interviewed IRS officials.
What GAO Found:
IRS has identified 20 service, compliance, and other benefits for the
first phase of CADE 2, including increasing the percentage of refunds
processed daily and reducing the number of erroneous notices due to
better account information, and has set quantitative targets for most
of these benefits. GAO has previously reported that quantitative
targets can be useful for tracking program performance. While it may
not always be possible to quantify targets, doing so helps to
objectively measure the extent to which expected benefits have been
realized. However, IRS has not yet finalized expected benefits for the
second phase or set related quantitative targets, because, according
to officials, these are contingent upon yet to be made design and
funding decisions. Nevertheless, addressing the second phase‘s
benefits and related targets as the design is being considered could
influence design decisions and help identify early on how systems and
processes might be affected.
IRS reported preliminary life cycle cost estimates for the first two
phases of the CADE 2 program of about $1.3 billion through 2024. This
includes about $377 million for development and $922 million for
operations and maintenance. IRS‘s process for developing the
preliminary estimates was generally consistent with best practices.
However, the agency did not follow three practices intended to improve
the credibility of cost estimates. Specifically, IRS did not (1)
consistently document excluded costs or provide a rationale for
excluding them; (2) use inflation in calculating costs; and (3)
perform an analysis to examine the effects of changing ground rules
and assumptions. While IRS stated it would perform the analysis of
changing ground rules and assumptions in revised estimates to be
available by the completion of our audit, until the agency implements
all these practices its estimates may not be credible.
IRS‘s process for managing the risks associated with CADE 2 is
generally consistent with best practices. Through its process, IRS
identified significant risks facing CADE 2, including that filing
season and other top information technology investment priorities may
result in contention for key resources, the delivery of the first
phase of CADE 2 may be delayed if deficiencies identified in
requirements are not corrected in a timely manner, and the risk that
technical challenges and other risks to implementing the database
identified as a result of prototyping efforts may not be addressed. To
its credit, IRS has developed mitigation strategies for each
identified risk. While IRS is working to ensure CADE 2 is successfully
managed, the schedule for delivering the initial phase is nevertheless
ambitious. IRS officials have acknowledged this and are taking actions
to increase their chances of meeting it, including moving certain
activities up, performing others concurrently, and adding checkpoints
to monitor the program‘s status. While these actions may increase the
likelihood of meeting the schedule, some of them, such as performing
activities concurrently, could potentially introduce more risk to
CADE 2‘s successful development and implementation.
What GAO Recommends:
GAO‘s recommendations include (1) identifying all of the second phase
benefits, setting the related targets, and identifying how systems and
business processes might be affected; and (2) improving the
credibility of revised cost estimates by including all costs or
providing a rationale for excluded costs, and adjusting costs for
inflation. In its comments on a draft of this report, IRS agreed with
GAO‘s recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-11-168] or key
components. For more information, contact David A. Powner at 202-512-
9286 or pownerd@gao.gov or James R. White at 202-512-9110 or
whitej@gao.gov.
[End of section]
Contents:
Letter:
Background:
IRS Has Identified Benefits and Set Most Related Targets for the First
Phase but Has Yet to Do So for the Second Phase:
IRS's Estimates the First Two Phases of CADE 2 Will Total About $1.3
Billion; Process for Developing Estimates Was Largely Consistent with
Best Practices:
IRS's Risk Management Process Is Generally Consistent with Best
Practices:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Additional Information about CADE 2 Benefits:
Appendix III: CADE 2 Program Risks:
Appendix IV: Comments from the Internal Revenue Service:
Appendix V: GAO Contacts and Staff Acknowledgments:
Tables:
Table 1: IRS's Plans for Addressing CADE Risks:
Table 2: Overview of CADE 2 Transition States:
Table 3: Number of IRS Quantified CADE 2 Benefits in TS1:
Table 4: Summary of CADE 2 Benefits IRS Expects Beyond TS1 and Number
of Related Quantitative Targets:
Table 5: Preliminary Estimates for the Eight Categories of Work for
CADE 2:
Table 6: Top CADE 2 Risks:
Table 7: CADE 2 Benefits:
Table 8: IRS Reported CADE 2 Risks and Mitigation Strategies:
Figure:
Figure 1: IRS's Preliminary Estimates of Annual Costs for CADE 2 (not
adjusted for inflation):
Abbreviations:
CADE: Customer Account Data Engine:
CMMI: Capability Maturity Model Integration:
IDRS: Integrated Data Retrieval System:
IMF: Individual Master File:
IPM: Integrated Production Model:
IRS: Internal Revenue Service:
IT: information technology:
MITS: Modernization and Information Technology Services:
PMO: Program Management Office:
TIGTA: Treasury Inspector General for Tax Administration:
TS1: transition state 1:
TS2: transition state 2:
W&I: Wage and Investment:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
March 24, 2011:
The Honorable José E. Serrano:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
House of Representatives:
Dear Mr. Serrano:
In administering federal tax laws, the Internal Revenue Service (IRS)
collected approximately $2.35 trillion in 2010 and is projected to
collect over $3 trillion beginning in 2013. To do so, IRS depends
heavily on information technology (IT) systems, including legacy tax
processing systems that were developed in the 1960s that are complex
and difficult to maintain.
In 1999, IRS initiated the Customer Account Data Engine (CADE) project
to replace legacy systems for storing, managing, and accessing
taxpayer accounts. Starting in 2005, CADE processed and recorded tax
return and tax account information for millions of individual
taxpayers with simple returns. However, IRS found that the development
of the system was more complex and taking longer than initially
anticipated and decided to undertake a broad review of its IT systems
modernization efforts. The results of the review led IRS to adopt a
new strategy for managing individual taxpayer accounts that would
provide benefits to IRS and the taxpayers sooner than CADE. The new
strategy is referred to as CADE 2.
To better understand the implications of moving to the new strategy,
you asked us to (1) determine whether IRS has identified the expected
benefits of CADE 2 and set targets for measuring success, (2) examine
the estimated costs of the program and assess IRS's process for
developing them, and (3) assess IRS's process for managing the risks
associated with CADE 2 and describe the risks IRS has identified using
this process.
To describe the expected benefits, we reviewed relevant documents,
including the CADE 2 Program Business Case and Benefits Management
Plan, and interviewed IRS officials from the offices of Wage and
Investment (W&I) and Modernization and Information Technology Services
(MITS). We also used program performance criteria established in our
prior reports. To understand the estimated costs and assess IRS's
process for developing them, we reviewed agency cost documentation and
compared it to the best practices identified in the GAO Cost
Estimating and Assessment Guide and interviewed officials from IRS's
Estimation Program Office. To determine CADE 2 key risks and IRS's
management of them, we interviewed officials from the CADE 2 Program
Management Office (PMO) and reviewed documentation associated with the
risk management process and the identification of key risks. We
evaluated IRS's risk management process against relevant criteria in
the Software Engineering Institute's Capability Maturity Model
Integration (CMMI) best practice guidance.
We conducted this performance audit from January 2010 to March 2011,
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives. Appendix I
contains further details on our objectives, scope, and methodology.
Background:
IRS's Business Systems Modernization program, which began in 1999, is
a multibillion-dollar, high-risk,[Footnote 1] highly complex effort
that involves the development and delivery of a number of modernized
tax administration and internal management systems, as well as core
infrastructure projects, which are intended to replace the agency's
aging business and tax processing systems and provide improved and
expanded service to taxpayers and internal business efficiencies for
IRS. CADE, one of the core systems of the Business Systems
Modernization program, was intended to provide a modernized system of
taxpayer accounts, with the ultimate goal of eventually replacing the
Individual Master File (IMF), a 1960s legacy tax master file that
maintains all taxpayer records for individual taxpayers.[Footnote 2]
The IMF is the authoritative data source for individual tax account
data. Tax data and related information pertaining to individual
taxpayers are posted to the IMF, and the IMF is updated annually to
incorporate new tax law procedures and changes. The IMF is a critical
component of IRS's ability to process tax returns, as all the other
IRS information system applications use data from this source.
According to IRS, the IMF is a complex computer system that maintains
all individual taxpayer records but has inherent limitations that
significantly constrain IRS's ability to achieve its mission. The
delays in weekly updates of taxpayer accounts, the lack of
synchronization of taxpayer data across dozens of systems, the
complexity of the data, the large volumes of data, and the ongoing
need to embed new tax laws and business rules contribute to IRS
systems storing, sharing, and processing data that is often
incomplete, untimely, or inaccurate.[Footnote 3]
With CADE, IRS started establishing a database and related
functionality for posting, settlement, maintenance, refund processing,
and issue detection for taxpayer account and return data for
relatively simple tax returns for individual taxpayers. Since 2005,
CADE has processed and recorded tax return and tax account information
for increasing numbers of individual taxpayers. IRS reported that, in
2010, CADE:
* processed about 41 million relatively simple returns (about 30
percent of the total individual income tax returns received); and:
* issued about 36 million refunds that totaled in excess of $66
billion (approximately 14 percent of total refunds), and processed
over 7 million taxpayer payments amounting to approximately $90
billion--with zero defects in balancing.[Footnote 4]
In addition, IRS reported that it issues refunds with CADE
approximately 5 days faster than refunds processed by the IMF.
Challenges and Risks Facing CADE That Led to CADE 2:
Although IRS made progress in processing individual taxpayer returns
with CADE, IRS found that each successive release of the system was
far more difficult as more complex accounts were to be supported,
raising concerns about the effort required to address these increasing
complexities. Also, since the CADE system operates concurrently with
IMF processing, work must be done across the two environments,
increasing the complexity of filing season operations and resulting in
a greater risk for errors in processing taxpayer account information.
IRS was concerned that, without a significant change in approach, CADE
would not be completed until at least 2020.
These challenges led the IRS Commissioner to form a team of senior IRS
technologists and external advisers to review CADE and IRS's
modernization strategy for individual taxpayer accounts. The review
identified six risks with the CADE strategy. As a result of the
review, in August 2008, IRS began defining a new strategy to address
the challenges confronting CADE and deliver improved individual tax
processing capabilities sooner. Specifically, the new strategy called
for accelerating completion of a modernized database and converting to
a single processing system sooner than the current approach would
allow. The new strategy is referred to as CADE 2. Table 1 describes
the CADE risks reported by IRS and how CADE 2 intends to address them.
Table 1: IRS's Plans for Addressing CADE Risks:
CADE risks: Lack of a target-state architecture to guide design
decisions leading to changes in each release of CADE that increased
rework to complete migration from IMF to CADE;
CADE 2 program response: IRS is expected to define and manage a target-
state architecture for managing individual taxpayer accounts. This
target-state solution architecture is to define the application
architecture, data architecture, operational flows, and standards for
the program and provides a clear definition of what it means to be
done.
CADE risks: Increasing risk of delivering the most complex business
functionality that would extend final delivery of CADE solution to
2020 or later;
CADE 2 program response: IRS has developed a transition strategy that
is expected to deliver the most complex data and business
functionality in early releases. The program is also using prototypes
(see the following paragraph) to prove out complex technical and
performance concepts early in the program and provide the necessary
guidance and information needed for detailed design.
CADE risks: Increasing filing season implementation complexity that
requires costly, complex transitional integration software for
exchanging account data between CADE and IMF and maintaining data in
both CADE and IMF formats;
CADE 2 program response: IRS has defined a set of transition states
that are expected to reduce the complexity of integration and
operation and maintenance of dual systems. The transition strategy is
expected to reduce filing season complexity by accelerating migration
to a single system for managing individual taxpayer accounts.
CADE risks: Increasing delivery effort and cost for each filing season
because IMF and CADE must run in parallel until migration of accounts
off of IMF to CADE is complete. IRS must design, develop, integrate,
and test system modifications needed to address tax law changes each
filing season (i.e., each year) in both systems;
CADE 2 program response: By accelerating migration to a single system
for managing individual taxpayer accounts, the CADE 2 program
transition strategy is expected to reduce the effort and cost
associated with implementing filing season changes.
CADE risks: Integration challenges with increased risks of breakdowns
as more complex taxpayer accounts are added to CADE since each new
release requires integration across more systems and processes than
prior releases;
CADE 2 program response: The CADE 2 PMO was established to address and
manage the integration challenges of building state-of-the-art
applications and databases, leveraging functionality embedded in
current systems and integrating each release of the solution with
upstream and downstream systems. It is expected that the PMO will
engage the appropriate IRS business partners and MITS delivery
partners, as well as contractors, to manage this complex transition
and balance the needs for successful delivery each filing season.
CADE risks: Critical shortage of IRS staff with in-depth business and
technical knowledge of IMF and CADE, both of which have to be
maintained until the migration is complete. These staff work not only
on CADE 2, but assist with other responsibilities, including IRS's tax
filing season operations and other Business Systems Modernization
projects;
CADE 2 program response: The PMO is expected to (and according to IRS,
has begun to) establish the partner and contractual relationships
required to manage and leverage the limited number of staff that
possess the in-depth business and technical knowledge of IMF, CADE,
and CADE 2. Changes in the relationships between the PMO and delivery
partners are expected to increase flexibility to leverage resources
with the critical IRS expertise and provide more IRS employees with
the specialized skills needed to deliver and maintain the CADE 2
solution.
Source: IRS.
[End of table]
As noted in the table, IRS used prototypes to test and prove complex
technical and performance concepts early and provide guidance and
information needed for detailed design.[Footnote 5] These prototypes
were completed in 2010, although some remaining tasks in one of them--
the Database Performance Test Prototype--will be completed later.
[Footnote 6]
CADE 2 Is Expected to Deliver Functionality in Three Phases
(Transition States):
CADE 2 is expected to deliver its functionality incrementally through
three phases known as transition states. Table 2 shows the target
completion dates and key characteristics for transition state 1 (TS1),
transition state 2 (TS2), and the target state.
Table 2: Overview of CADE 2 Transition States:
Transition states and target completion dates: TS1; January 2012;
Key characteristics of CADE 2 transition states: Dual Systems--IMF and
CADE;
* daily batch processing of individual taxpayer returns provided by
modifying the IMF to run on a daily, rather than weekly, basis;
* comprehensive database established for housing all individual
taxpayer accounts and loaded with data from CADE and IMF;
* database provides timelier updates of taxpayer information for use
by IRS employees for compliance and customer service.
Transition states and target completion dates: TS2; January 2014;
Key characteristics of CADE 2 transition states: Single System--CADE;
* target technology developed and deployed (single processing system;
IMF retired);
* high-priority downstream service and compliance applications
modified to take advantage of the new database;
* some key financial material weaknesses and applications addressed.
Transition states and target completion dates: Target state; TBD;
Key characteristics of CADE 2 transition states: Single System--CADE;
* complete the transition of applications that use the target database
so downstream systems fully leverage the database;
* address all financial and security material weaknesses identified at
the inception of the program[A];
* eliminate transitional components that were required during the
transition states.
Source: GAO analysis of IRS data.
[A] We have previously reported that one significant weakness is IRS's
inability to distinguish between types of unpaid tax assessments (GAO,
Financial Audit: IRS's Fiscal Years 2010 and 2009 Financial
Statements, GAO-11-142 (Washington, D.C., Nov. 10, 2010)). An unpaid
assessment is a legally enforceable claim against a taxpayer and
consists of taxes, penalties, and interest that have been assessed to
the taxpayer but not yet collected or abated (reduced). Since these
assessments involve the Business Master File, and this system remains
outside the scope of CADE 2, IRS will therefore be unable to fully
address all financial material weaknesses with CADE 2.
[End of table]
IRS Has Established Two Projects to Achieve TS1 Goals:
TS1 consists of the following two projects:
Daily Processing: According to IRS, this project is to enable IRS to
process and post all individual taxpayer returns filed and other
transactions by updating and settling individual taxpayer accounts in
24-48 hours with current, complete, and authoritative data, and
provide employees with timely access.[Footnote 7] More specifically,
through the Daily Processing project, IRS is expected to:
* modify the IMF to run daily, i.e., modify current transaction
processing to occur daily versus weekly;
* make changes to the Integrated Data Retrieval System (IDRS)[Footnote
8] and the notice generation process to support the daily vs. weekly
processing cycle; and:
* prepare daily "loads" to update the CADE 2 database for IDRS and
other downstream systems as they are able to support daily processing.
Database Implementation: According to IRS, this project is to
establish the CADE 2 database, a relational database that will house
data on individual taxpayers and their accounts; develop a capability
to transfer data from IMF to the database; and provide for the
transfer of data from the database to downstream IRS financial,
customer service and compliance systems. Specifically, the project is
expected to enable IRS to:
* perform a one-time initialization of the database when TS1 begins
(this will involve loading all taxpayer account data from IMF and the
CADE database into the CADE 2 database and then validating it);
* perform daily updates to the database after TS1 begins by
extracting, transforming, and loading daily processing changes to IMF
data;[Footnote 9] and:
* utilize the database to provide individual taxpayer account
information to selected downstream systems to assist IRS service and
compliance personnel. For example, the CADE 2 database is intended to
provide daily updates to IDRS, as well as weekly updates to the
Integrated Production Model (IPM)--a downstream data repository
designed to support IRS compliance functions that house IMF and
Business Master File data, information returns, and other data. IPM is
expected to include current individual taxpayer data from the CADE 2
database.
Regarding the status of these two projects, IRS completed efforts to
define the TS1 projects' logical design in December 2010 and began the
detailed design phase, which includes activities such as documenting
the physical design of the solution. IRS expects to complete this
phase by April 2011.
IRS has also defined overall objectives for TS2 and the target state,
but detailed planning of TS2 is in the early stages, with the
formation of a planning team that expects to define a high-level
approach by early May 2011.
Treasury Inspector General for Tax Administration and GAO Have
Recently Reported on CADE 2:
In December 2009, the Treasury Inspector General for Tax
Administration (TIGTA) identified several challenges that IRS needed
to address to effectively manage identified CADE 2 risks, including,
among other things, implementing a governance structure for the PMO to
provide oversight and direction for the implementation of CADE 2 and
developing contingency plans in the event that CADE 2 cannot be fully
implemented.[Footnote 10] IRS officials completed a contingency
strategy in November 2010.
In May 2010, we reported that, while much had been done to define CADE
2's transition states, IRS had not identified time frames for
completing key planning activities for TS2.[Footnote 11] Consequently,
we recommended IRS establish these time frames. In response, IRS
stated that it would develop a plan for launching TS2 activities that
will outline the approach and a high-level plan for the transition
state. IRS also noted that business requirements would be completed in
parallel with the development of the plan. In December 2010, IRS
initiated TS2 planning activities and set a target of May 2011 for
agreeing on the scope and plan for the transition state.
In November 2010, TIGTA reported on the prototypes undertaken by IRS
to gain confidence in the CADE 2 solution.[Footnote 12] TIGTA found
that the five prototype teams that were established generally managed
their objectives effectively, and that the teams also identified risks
that faced the successful execution of the prototype plans and took
steps to overcome the barriers. TIGTA recommended that IRS take
several actions to reemphasize compliance with the elements of the
CADE 2 Prototype Process. In its response, IRS agreed with TIGTA's
recommendations and described steps to address them.
IRS Has Established a New Management Structure for the CADE 2 Program:
While the MITS organization has primary responsibility for developing,
managing, and delivering the CADE 2 program, IRS has established a new
governance approach for the program, including the CADE 2 PMO that is
to manage the program and the relationship with the program's
stakeholders, define the solution specifications to meet the
requirements and the delivery of the program scope, and specify the
roles and responsibilities of all parties. The new governance approach
is intended to foster rapid decision-making and proactive risk
management and issue resolution, as well as ensure that the
appropriate stakeholders are involved, engaged, and collaborating as a
unified team in making CADE 2 decisions.
The CADE 2 PMO is headed by an Associate Chief Information Officer and
made up of several groups, each responsible for a different aspect of
the program. These groups include:
* Delivery Management. Responsible for overseeing and coordinating
activities across the projects that are implementing the solution for
each of the transition states.
* Program Management and Control. Responsible for executing the
program management processes, CADE 2 solution planning, and governance
and control activities.
* Chief Architect. Responsible for setting the overall technical
direction in alignment with IRS's business and technical architecture,
as defined in the enterprise architecture.[Footnote 13]
* Chief Engineer. Responsible for reviewing detailed design and
interfaces to ensure the CADE 2 solution will work, is secure, and
integrates with other systems and infrastructure.
* Business Operations. Responsible for providing the administrative
support for all branches of the PMO.
These groups are to work with (1) delivery partners who are other MITS
organizations (e.g., Applications Development and Enterprise
Operations) that work to deliver the program scope, (2) business
partners (e.g., W&I and the Chief Financial Officer) who are
organizations who either use the system or define the business
requirements and, as such, play a key role in overseeing CADE 2, and
(3) other stakeholder groups who support, influence, or oversee the
program.
To oversee and guide the PMO, IRS also established several executive-
level boards and advisory councils composed of business and IT
officials. The executive-level boards include the following:
* The CADE 2 Executive Steering Committee, which consists of senior
executives from MITS, W&I, and Department of the Treasury, and serves
as an oversight group that ensures the program stays aligned with the
IRS strategic plan and approves decisions with significant
organizational or external impact; and:
* The CADE 2 Governance Board, which consists of Associate Chief
Information Officers from CADE 2 and Applications Development and the
business modernization executive from W&I, and ensures that objectives
are met; decisions and issues are resolved in a timely manner; risks
are managed appropriately; and the expenditure of resources allocated
is fiscally sound. The CADE 2 Governance Board also approves program
risk response plans, milestone exits, and resolves escalated issues.
The advisory councils include the:
* Executive Oversight Team, which consists of executives from the CADE
2 PMO, delivery partners, and senior-level business executives, and
serves as the oversight for the day-to-day execution of the CADE 2
program;
* Associate Chief Information Officer Advisory Council representing
leaders of each of the MITS technical units;
* Program Leadership Advisory Council, a broad group of executives
from W&I, Agency Wide Shared Services, and across MITS that consult on
key issues, risks, analyses, and recommendations on an as needed
basis; and:
* Architecture/Engineering Review Council recently created to provide
architectural and engineering leadership across the program to ensure
design and development efforts are complete and adhere to solution
architecture and engineering best practices.
In addition, while the projects chartered under CADE 2 are expected to
follow the IRS enterprise life cycle[Footnote 14] methodology that
other projects are required to follow, IRS also defined a program
management framework for CADE 2 that defines the phases, activities,
deliverables, milestones, and reviews necessary to manage both the
program and each of its component projects in a coordinated and
integrated manner. The PMO and supporting boards are to play key roles
in reviewing deliverables and approving milestone exits for the
program and assuring that the projects are properly aligned and
integrated with the program.
Finally, IRS has also defined and started implementing several
processes to support its management of the CADE 2 program. They
include processes for managing risks, the integrated master schedule,
the prototype process, and other activities. While several of these
processes were already established for projects, IRS enhanced them for
CADE 2 to take into account the complexities associated with
integrating projects into a program.
IRS Has Identified Benefits and Set Most Related Targets for the First
Phase but Has Yet to Do So for the Second Phase:
For TS1, IRS has identified 20 benefits. These benefits span three
categories: service; compliance; and other benefits, including reduced
system costs and improved security benefits.
Our past work at IRS established that quantitative targets can be
useful for tracking program performance.[Footnote 15] While it may not
always be possible to quantify targets, doing so helps to objectively
measure the extent to which expected benefits have been realized.
As shown in table 3, IRS has set quantitative targets for 15 of the 20
benefits for TS1. (Table 7 in app. II describes all the benefits IRS
has identified and the related quantitative targets where they exist.)
Table 3: Number of IRS-Quantified CADE 2 Benefits in TS1:
Benefit category: Service benefits;
Benefit subcategory: Faster refunds and processing;
Benefits with quantitative targets in TS1: 4 of 4.
Benefit category: Service benefits;
Benefit subcategory: Faster payment posting;
Benefits with quantitative targets in TS1: 1 of 1.
Benefit category: Service benefits;
Benefit subcategory: More efficient account adjustment;
Benefits with quantitative targets in TS1: 1 of 1.
Benefit category: Service benefits;
Benefit subcategory: Reduction in manual refunds;
Benefits with quantitative targets in TS1: 0 of 1.
Benefit category: Service benefits;
Benefit subcategory: Updated account information;
Benefits with quantitative targets in TS1: 1 of 2.
Benefit category: Service benefits;
Benefit subcategory: Faster notices;
Benefits with quantitative targets in TS1: 1 of 1.
Benefit category: Service benefits;
Benefit subcategory: Systematic resolution;
Benefits with quantitative targets in TS1: 1 of 1.
Benefit category: Compliance benefits;
Benefit subcategory: Decreasing taxpayer overpayment;
Benefits with quantitative targets in TS1: 0 of 1.
Benefit category: Other benefits;
Benefit subcategory: Reduced system maintenance costs;
Benefits with quantitative targets in TS1: 4 of 4.
Benefit category: Other benefits;
Benefit subcategory: Improved security;
Benefits with quantitative targets in TS1: 2 of 4.
Benefit category: Total;
Benefits with quantitative targets in TS1: 15 of 20.
Source: GAO analysis of IRS data.
[End of table]
An example of a benefit with a quantitative target is the percentage
of refunds IRS will process on a daily basis. IRS expects CADE 2 to
increase this from 30 to 80 percent, resulting in faster refunds to
many taxpayers. Another example is the security benefit of reducing
the number of IRS databases that contain extracts of data from CADE 2.
IRS expects this to be reduced by 20 percent from the 2011 baseline.
IRS officials have stated that they have not set quantitative targets
for five TS1 benefits for two reasons. First, officials told us that
two benefits related to improved security and updated account
information cannot be quantified and therefore do not have numeric
targets. (See appendix II for more details.) Second, IRS officials
stated that competing priorities have impeded their ability to define
the remaining targets. While they stated they plan to define these
targets, they have not committed to specific time frames for doing so.
Until IRS has set quantitative targets where feasible in TS1, it and
other stakeholders including Congress will not have complete
information on the specific benefits that can be expected from funds
spent on this phase. In addition, it will be more difficult to
objectively measure whether expected benefits have been delivered.
In contrast with TS1, IRS has not finalized the benefits expected in
TS2 or defined related quantitative targets. Specifically, IRS has
identified a list of 22 service, compliance, and other benefits
including addressing financial material weaknesses it expects to
achieve beyond TS1.[Footnote 16] These benefits are summarized in
table 4 and described in greater detail in table 7 in appendix II.
However, IRS has not yet determined whether these benefits are to be
delivered in TS2 or the target state. According to IRS, committing to
what can be delivered in TS2 is contingent upon a number of decisions
yet to be made. Specifically, officials stated that IRS must first
finalize its approach for TS2 (as noted earlier, the agency expects to
do this by May 2011). Officials also stated that funding decisions for
the upcoming fiscal year could impact IRS's plans for TS2.
In addition, IRS has not finished defining the full range of possible
benefits it expects beyond TS1. For example, one possible compliance
benefit that IRS is exploring is requiring providers of information
returns to send those returns to IRS and taxpayers at the same time,
which would allow IRS to use this data earlier to conduct some
compliance checks.[Footnote 17] Conducting earlier analyses of tax
return information using CADE 2 and IPM could allow IRS to stop an
erroneous refund before it goes out. IRS currently waits until well
after the filing season to conduct some compliance checks, such as
computerized matching of information returns and tax returns and
examinations of taxpayers' books and records. Such compliance checks
can lead to costly collection efforts from noncompliant taxpayers.
Finally, as summarized in table 4 and shown in greater detail in
appendix II, IRS has not yet set quantitative targets for any of the
benefits defined or committed to any time frames for doing so. As
noted for TS1, until IRS has set quantitative targets where feasible
in TS2, it and other stakeholders will not have complete information
on the specific benefits than can be expected from funds spent, and it
will be difficult to objectively measure whether expected benefits
have been delivered.
Table 4: Summary of CADE 2 Benefits IRS Expects Beyond TS1 and Number
of Related Quantitative Targets:
Benefit category: Service benefits;
Benefit subcategory: More efficient account adjustment;
Benefits with quantitative targets beyond TS1: 0 of 1.
Benefit category: Service benefits;
Benefit subcategory: Updated account information;
Benefits with quantitative targets beyond TS1: 0 or 1.
Benefit category: Service benefits;
Benefit subcategory: Systematic resolution;
Benefits with quantitative targets beyond TS1: 0 of 3.
Benefit category: Compliance benefits;
Benefit subcategory: Increasing taxpayer compliance;
Benefits with quantitative targets beyond TS1: 0 of 3.
Benefit category: Compliance benefits;
Benefit subcategory: Increasing tax preparer compliance;
Benefits with quantitative targets beyond TS1: 0 of 2.
Benefit category: Compliance benefits;
Benefit subcategory: Improved collection process;
Benefits with quantitative targets beyond TS1: 0 of 1.
Benefit category: Compliance benefits;
Benefit subcategory: Improved audit case selection;
Benefits with quantitative targets beyond TS1: 0 of 3.
Benefit category: Other benefits;
Benefit subcategory: Reduced system maintenance costs;
Benefits with quantitative targets beyond TS1: 0 of 3.
Benefit category: Other benefits;
Benefit subcategory: Addressed financial material weakness;
Benefits with quantitative targets beyond TS1: 0 of 5.
Benefit category: Total;
Benefits with quantitative targets beyond TS1: 0 of 22.
Source: GAO analysis of IRS data.
[End of table]
We acknowledge that thinking through the expected TS2 benefits
(including those from the table in app. II and other potential
benefits) and related targets as the approach and design are being
considered may result in having to make adjustments later. However,
the benefits and targets generated at this stage can influence design
decisions and allow IRS to identify early how associated systems and
business processes might be affected.
IRS's Estimates the First Two Phases of CADE 2 Will Total About $1.3
Billion; Process for Developing Estimates Was Largely Consistent with
Best Practices:
In July 2009, IRS reported preliminary life cycle cost estimates for
TS1 and TS2 of about $1.3 billion through 2024, including about $377
million for development and $922 million for operations and
maintenance. The estimates include costs for eight categories of work
needed to achieve the goals defined for TS1 and TS2. The majority of
the costs are for the Operational Framework category ($663 million or
51 percent of total costs), which is to provide for the infrastructure
on which to develop, test, and deploy the new and enhanced
applications under the CADE 2 program. The CADE 2 preliminary cost
estimates are summarized in table 5.
Table 5: Preliminary Estimates for the Eight Categories of Work for
CADE 2:
Cost category: 1. Program office;
Development cost: $65 million;
Operations and maintenance cost: $0;
Total cost: $65 million.
Cost category: 2. Current CADE enhancements;
Development cost: $72 million;
Operations and maintenance cost: $55 million;
Total cost: $127 million.
Cost category: 3. IMF enhancements;
Development cost: $42 million;
Operations and maintenance cost: $62 million;
Total cost: $104 million.
Cost category: 4. Database development (TS1);
Development cost: $15 million;
Operations and maintenance cost: $20 million;
Total cost: $35 million.
Cost category: 5. Database development (TS2);
Development cost: $7 million;
Operations and maintenance cost: $7 million;
Total cost: $13 million.
Cost category: 6. Operational framework (TS1 and TS2);
Development cost: $72 million;
Operations and maintenance cost: $591 million;
Total cost: $663 million.
Cost category: 7. Database access;
Development cost: $81 million;
Operations and maintenance cost: $133 million;
Total cost: $214 million.
Cost category: 8. Financial settlement;
Development cost: $24 million;
Operations and maintenance cost: $54 million;
Total cost: $78 million.
Cost category: Total;
Development cost: $377 million;
Operations and maintenance cost: $922 million;
Total cost: $1.299 billion.
Source: IRS.
Note: IRS did not adjust these costs for inflation.
[End of table]
The estimated annual costs from 2009 through 2024 are shown in figure
1. At the time they were developed, IRS estimated CADE 2 development
costs to end in 2015 and total costs to peak during 2011 at about $162
million. Estimated costs between 2015 and 2024 are all recurring
operations and maintenance costs.
Figure 1: IRS's Preliminary Estimates of Annual Costs for CADE 2 (not
adjusted for inflation):
[Refer to PDF for image: vertical bar graph]
Year: 2009;
Development costs: $43.8 million;
Operations and Maintenance costs: $14.9 million.
Year: 2010;
Development costs: $103.1 million;
Operations and Maintenance costs: $16 million.
Year: 2011;
Development costs: $98.7 million;
Operations and Maintenance costs: $63.2 million.
Year: 2012;
Development costs: $70.3 million;
Operations and Maintenance costs: $69.9 million.
Year: 2013;
Development costs: $44.1 million;
Operations and Maintenance costs: $72.55 million.
Year: 2014;
Development costs: $17 million;
Operations and Maintenance costs: $83.35 million.
Year: 2015;
Operations and Maintenance costs: $86.65 million.
Year: 2016;
Operations and Maintenance costs: $86.65 million.
Year: 2017;
Operations and Maintenance costs: $86.65 million.
Year: 2018;
Operations and Maintenance costs: $86.65 million.
Year: 2019;
Operations and Maintenance costs: $86.65 million.
Year: 2020;
Operations and Maintenance costs: $86.45 million.
Year: 2021;
Operations and Maintenance costs: $45.55 million.
Year: 2022;
Operations and Maintenance costs: $18.6 million.
Year: 2023;
Operations and Maintenance costs: $14 million.
Year: 2024;
Operations and Maintenance costs: $3.3 million.
Source: GAO analysis of IRS data.
[End of figure]
IRS's estimates did not include expected costs for the following
items:[Footnote 18]
* operations and maintenance of the IMF system to include enhancements
needed to achieve the CADE 2 target state,
* prototype development and assessments,
* technical training (estimated at $1 million to $3 million),
* work required to reach the final target state, and:
* process reengineering costs/business costs associated with
implementing the current weekly batch processing of accounts to the
daily processing required at the completion of TS1.
With Some Exceptions, Process for Developing Preliminary Cost
Estimates Was Consistent with Best Practices:
The process IRS used to develop the CADE 2 preliminary cost estimates
was generally consistent with the best practices outlined in the GAO
cost estimating guide.[Footnote 19] Specifically, IRS followed a well-
defined process for developing a comprehensive, documented, accurate,
and mostly credible life cycle cost estimate for the first two phases
of CADE 2. For example, the estimation process was performed in
accordance with a well-defined plan by an independent Estimation
Program Office. Parametric estimation models were used that contained
catalogs based on historical data, along with cost estimating
relationships, designed to produce estimates analogous to similar
programs. In addition, IRS used simulation techniques to incorporate
risks and determine confidence bounds on the estimates.
Further, IRS plans to update its estimates, consistent with the best
practice to update estimates to reflect changes in technical or
program assumptions. Specifically, since the estimates were prepared,
IRS has developed a better understanding of the work required to meet
CADE 2 objectives for TS1 and the current work breakdown structure
(i.e., definition of work to be performed) no longer matches the one
used to develop the preliminary estimates. IRS officials told us that,
consistent with IRS's Estimation Program Office Estimator's Guide,
they plan to revise their estimates after detailed design information
for the database implementation and daily processing projects is
completed and expect them to be available by the completion of our
audit. CADE 2 PMO officials stated they did not expect the revised
estimates to differ significantly from the current estimate.[Footnote
20] It is important to note that, even when all TS1 cost elements have
been updated to accurately reflect the current understanding of the
work to be performed, the total cost of ownership through TS2 could
still significantly differ from the new estimate, given that, as
previously noted, the approach for TS2 has yet to be defined.
We identified the following three practices IRS did not follow that
could strengthen its methods for estimating CADE 2 costs to ensure
they are more credible:
First, according to best practices, items excluded from estimates
should be documented and explained. While IRS documented four of the
five items that were excluded from the estimates, it did not provide
an explanation for excluding these costs. In addition, IRS did not
document that it excluded the business costs associated with moving to
daily processing. IRS officials stated that, at the time the estimates
were developed, these business costs were not well understood by the
estimators and now believe that these costs would be relatively small,
representing mostly costs for staff already being paid out of existing
budgets. Nevertheless, consistently documenting excluded costs and
providing a rationale for excluding them would improve credibility.
Second, IRS's estimates did not include inflation. According to best
practices, adjusting costs for inflation correctly is necessary if the
cost estimate is to be credible. Inflation reflects the fact that the
cost of an item usually continues to rise over time. If a mistake is
made in applying inflation, or if inflation is not included, cost
overruns can result.
Third, IRS did not conduct a sensitivity analysis to examine the
effects of changing assumptions and ground rules on its estimates.
While IRS developed risk-adjusted estimates, it did not perform the
formal sensitivity analysis necessary to understand which variables
most affect the cost estimate. IRS officials stated they intend to
include a sensitivity analysis in the revised cost estimates expected
to be available by the completion of our audit. Performing this
analysis will not only improve the credibility of the estimate, but it
will provide management a better understanding of the relationship of
key factors to cost, which is important in evaluating different
options as part of subsequent risk and issue mitigation strategies and
contingency plans.
IRS's Risk Management Process Is Generally Consistent with Best
Practices:
Given the size and significance of the CADE 2 program, it is important
that IRS have an effective risk management process. According to best
practices, an effective risk management process identifies potential
problems before they occur, so that risk-mitigating activities may be
planned and invoked as needed across the life of the product and
project in order to mitigate adverse impacts on achieving objectives.
It includes assigning resources, identifying and analyzing risks, and
developing risk mitigation plans and milestones for key mitigation
deliverables.[Footnote 21]
IRS's risk management process for CADE 2 is generally consistent with
best practices. For example, regarding resources assigned to risk
management, the CADE 2 PMO's Director for Program Management and
Control is responsible for "executing" the risk management process.
Biweekly meetings are held where the Director for Delivery Management
and project officials discuss project status and risks and determine
whether risks need to be escalated as candidate risks to the program
level. Monthly risk and issue meetings are held to discuss the program-
level risks. Finally, those risks that require additional oversight
are then presented to the Chief Technology Officer for review. Program-
level and project-level risks are scored and tracked in a consistent
manner using a risk repository tool. Risk mitigation plans and
associated milestones are defined for each risk. Officials from the
PMO have also told us that the management structure and processes that
have been established for the program are intended to help mitigate
the risks associated with developing such a large and complex program.
While IRS is generally effectively carrying out the activities
associated with the risk management process, the two key documents
that support this process are in some cases inconsistent and do not
fully reflect actions being taken. For example, the Risk and Issue
Management Process[Footnote 22] document calls for automatically
escalating those projects with a high to very high risk exposure score
[Footnote 23] to the next level of management while the Risk and Issue
Management Plan[Footnote 24] calls for escalating risks based
primarily upon the judgment of the risk owner and responsible manager.
The Risk and Issue Management Process identifies a specific set of
actions to mitigate risks depending on the risk exposure score while
the Risk and Issue Management Plan provides high-level guidance
describing various alternatives that can be taken to mitigate risks.
As another example, both documents identify the risk management roles
but do not specify when in the process these roles are assigned or
what the required qualifications are.
IRS recognizes its risk management procedures could be strengthened,
and the Director for Program Management and Control intends to review
these procedures by the completion of our audit, to ensure they are
consistent and clearly reflect the process in place. This action will
help IRS achieve optimal effectiveness in identifying, prioritizing,
or managing risks that could affect the cost and schedule of the
program.
IRS Has Identified Significant Risks Confronting CADE 2:
Using its risk management process, IRS has identified several
significant program risks for CADE 2. As of November 2010, six of
these were considered "top risks." These "top risks," which include
TS1 contingency planning and resource contention among IRS priorities
are listed in table 6 along with the mitigation strategies that have
been defined for them. (For a complete list of program risks, see
appendix III.)
Table 6: Top CADE 2 Risks:
Reported risk: Database implementation challenges;
IRS's description: The Database Implementation project faces many
challenges as the IRS moves toward establishing an authoritative
relational database. These challenges include a number of technical
hurdles and risks that were revealed as part of the recent extract,
transform, and load prototyping efforts;
Mitigation: Establishing a dedicated CADE Database Implementation
project team with end-to-end responsibility for delivery of the
database software component.
Reported risk: TS1 contingency planning;
IRS's description: If TS1 contingency planning and associated work
efforts are not addressed in a timely manner, the CADE 2 program may
not be able to exercise contingency options in the event that full TS1
is not ready for deployment in January 2012;
Mitigation: Presenting contingency options to governance boards and
updating contingency or deployment strategies/plans will be based on
decisions made by these boards.
Reported risk: Resource contention among IRS priorities;
IRS's description: Filing season and other top IT investment
priorities may result in resource contention for key resources;
Mitigation: Establishing a process for monitoring and updating the
CADE 2 Staffing Strategy and Plan that includes understanding
government staffing needs in Delivery Partner organizations.
Reported risk: Availability of FY 2011 funding;
IRS's description: It is critical that the CADE 2 program obtain
sufficient funds timely to successfully deploy the TS1 solution. If FY
2011 CADE 2 funding is not received within the first 3 months of FY
2011, the infrastructure for building out the production environment
and disaster recovery for TS1 development may be in jeopardy;
Mitigation: Making a deliberate decision to manage the risk through
financial operations control activities performed in conducting
business operations.
Reported risk: Requirements maturity;
IRS's description: The successful implementation of the CADE 2
solution is very dependent on a solid set of baselined requirements.
If the deficiencies identified in the current state of requirements
are not corrected timely, the delivery of CADE 2 TS1 may be delayed;
Mitigation: Identifying the list of requirement deficiencies and
assigning a working group to validate corrective actions, assigning
ownership, and developing an implementation plan for completing each
correction action.
Reported risk: Database extract, transform, and load prototype 2;
IRS's description: If the database extract, transform, and load
prototype results show that a component of the database process fails,
an alternative design solution will need to be recommended for the
timely completion of the design of the Database Implementation project;
Mitigation: The database extract, transform, and load work breakdown
structure has included tasks to validate the resulting data and to
perform analysis for a recommended course of action after each
iteration is complete. The recommendation for go or no-go decision
point is made based on that analysis performed.
Source: IRS.
[End of table]
While IRS has taken many steps to reduce the risks associated with
CADE 2, the agency will nevertheless be challenged in meeting its
January 2012 schedule for delivering the benefits associated with
daily processing and database implementation given the amount of work
needed to complete TS1 over the next year. This includes completing
logical design and detailed design activities, testing and integration
of core daily applications and their development, and testing and
population of the database with current CADE and IMF data. Further, as
already noted, waiting to finish (1) defining targets for the TS1
benefits and (2) finalizing benefits and related targets for TS2
creates a risk of costly or time-consuming late design changes.
Recognizing the challenges it faces, IRS has taken additional steps to
increase the likelihood of meeting the schedule. These include moving
certain activities up, performing others concurrently and adding
checkpoints to monitor the program's status. In addition, IRS has
recently completed a TS1 contingency strategy.[Footnote 25] While
these actions may increase the likelihood of meeting the schedule,
some of them, such as performing activities concurrently, could
potentially introduce more risk to CADE 2's successful development and
implementation.
Conclusions:
Defining benefits and related quantitative targets where feasible is
important to understanding what to expect and how to measure
performance against expectations. For TS1, while IRS has identified
numerous expected benefits, it has not yet defined quantitative
targets for all of them. Given that IRS is finalizing the design and
preparing to implement TS1, it is important for the agency to complete
defining the targets associated with TS1 benefits. Delaying the
setting of targets could result in costly and time-consuming changes
to the design and associated requirements. Similarly, addressing TS2
benefits and related targets as the design is being considered could
influence design decisions and help identify early on how systems and
processes might be affected.
In developing cost estimates for the first two phases of CADE 2, IRS
did not disclose all excluded costs, provide a rationale for
documented excluded costs, adjust for inflation, or perform a
sensitivity analysis to examine the effects of changing assumptions
and ground rules. IRS stated it would perform a sensitivity analysis
in the revised estimates it expects to release by the completion of
our audit.
To its credit, IRS has established a risk management process that is
generally consistent with best practices and has identified mitigation
strategies to address each identified risk. Several of the risks
identified are significant and will require continued attention from
IRS.
Recommendations for Executive Action:
We recommend that the Commissioner of Internal Revenue direct the
appropriate officials to take the following five actions:
* to provide a better basis for measuring CADE 2 performance in
achieving TS1 benefits, set the remaining quantitative targets where
feasible as soon as possible in 2011;
* in conjunction with developing the approach for TS2 expected in May
2011, finalize the phase's expected benefits, set quantitative targets
where feasible, and identify how related systems and business
processes might be affected; and:
* to ensure the revised estimates for CADE 2 are credible,
- include inflation when calculating costs,
- include the costs IRS explicitly excluded or provide a rationale for
excluding them, and:
- include any business costs associated with moving to daily
processing or document that these costs were excluded and provide a
rationale for excluding them.
Agency Comments and Our Evaluation:
IRS's Commissioner provided written comments on a draft of this report
(reprinted in app. IV.) In its comments, IRS stated that it
appreciated that the report recognized the progress the agency had
made and that it is following best practices on important disciplines
like risk management and cost estimation. IRS also agreed with our
recommendations and stated it would provide a detailed corrective
action plan addressing each one along with its response to our final
report. IRS also provided revised cost estimates for CADE 2 that
address our finding related to the use of a sensitivity analysis.
We are sending copies of this report to the appropriate congressional
committees, and to the Chairmen and Ranking Members of other Senate
and House committees and subcommittees that have appropriation,
authorization, and oversight responsibilities for IRS. We will also
send copies to the Commissioner of Internal Revenue, the Secretary of
the Treasury, the Chairman of the IRS Oversight Board, and the
Director of the Office of Management and Budget. The report also is
available at no charge on the GAO Web site at [hyperlink,
http://www.gao.gov].
If you or your staff members have any questions or wish to discuss the
material in this report further, please contact us at (202) 512-9286
or pownerd@gao.gov or at (202) 512-9110 or whitej@gao.gov. Contact
points for our Offices of Congressional Relations and Public Affairs
may be found on the last page of this report. GAO staff who made key
contributions to this report are listed in appendix V.
Sincerely yours,
Signed by:
David. A. Powner:
Director, Information Technology Management Issues:
Signed by:
James R. White:
Director, Tax Issues Strategic Issues Team:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
As agreed, our objectives were to:
1. determine whether the Internal Revenue Service (IRS) has identified
the expected benefits of Customer Account Data Engine (CADE) 2 and set
targets for measuring success,
2. examine the estimated costs of CADE 2 and assess IRS's process for
developing them, and:
3. assess IRS's process for managing the risks associated with CADE 2
and describe the risks IRS has identified using this process.
For our first objective, we reviewed relevant documents, including the
CADE 2 Program Business Case and Benefits Management Plan, and
interviewed officials from IRS offices: Wage and Investment and
Modernization and Information Technology Services (MITS). We also used
criteria from our prior work to identify attributes of successful
performance measures. We reviewed documents including the Program
Roadmap, Business Case and Benefits Management Plan, and Benefits
Roadmap, and the Office of Management and Budget Exhibit 300.
For our second objective, we summarized the estimated costs reported
in the July 2009 CADE 2 Estimate Summary/Basis of Estimate and totaled
costs by year for each of the major categories for which estimates
were developed. We reviewed several documents, including the July 2009
CADE 2 Estimate Summary/Basis of Estimate and the Estimation Breakdown
Summary reports and capabilities and solution concept documents (e.g.,
the Integrated Master File Enhancements: Capabilities Definition and
Solution Concept document and the Financial Settlement: Capabilities
Definition and Solution Concept document), which support the Basis of
Estimate. We also reviewed IRS's Estimators' Reference Guide and
aspects of the SEER-SEM methodology that was used to develop the
estimates. We also interviewed staff from the Estimation Program
Office, as well as members of the Program Management and Control
function of the CADE 2 Program Management Office (PMO) responsible for
tracking the program's cost. We compared what we learned with the 12
steps for developing well-documented, comprehensive, accurate, and
credible cost estimates identified in the GAO Cost Estimating and
Assessment Guide.[Footnote 26]
For our third objective, we reviewed documented procedures supporting
the risk management process, including the Risk and Issue Management
Process, the Risk and Issue Management Plan, the user guide for the
Item Tracking Reporting and Control system that is used to track risks
and the MITS Risk Identification Procedure. We also reviewed artifacts
of the risk management process, including program risk logs and
detailed risk reports. We participated in meetings in which risks were
discussed and reviewed agendas and meeting minutes for others. We also
interviewed several officials, including the CADE 2 Program Director,
the Director of the CADE 2 PMO Program Management and Control office,
and the Director of the CADE 2 PMO Delivery Management office. Through
document reviews and interviews with appropriate officials, we also
examined the following areas in greater depth: (1) schedule, (2)
contractor oversight, (3) requirements management, (4) prototypes, and
(5) human capital. We selected these areas because, based on our
experience, they represent areas of risk or have been problematic for
IRS in the past. In addition, we examined IRS's prototyping efforts to
understand how the agency was using them to mitigate risks. We
evaluated what we learned about IRS's risk management process against
relevant criteria for Software Engineering Institute's Capability
Maturity Model Integration (CMMI) to determine the effectiveness of
the process.
To obtain background information on the program, we reviewed a host of
documents developed by the CADE 2 PMO, including the CADE 2 Program
Charter, Solution Architecture, Program Roadmap, Program Management
Plan, the charters of the two CADE 2 transition state 1 projects
(Daily Processing and Database Implementation), as well as governance
team meeting minutes. We also observed CADE 2 meetings, conducted
interviews with and received briefings from CADE 2 officials,
including the IRS Chief Technology Officer, the Deputy Chief
Information Officer for Strategy and Modernization, the Associate
Chief Information Officer who serves as executive-in-charge of the
CADE 2 program, and the Chief Architect of the CADE 2 program.
We conducted this performance audit from January 2010 to March 2011
primarily at the New Carrollton Federal Building in New Carrollton,
Maryland, where the CADE 2 staff are located, and Atlanta, Georgia,
where the Wage and Investment division staff are located, in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
[End of section]
Appendix II: Additional Information about CADE 2 Benefits:
Table 7 shows a list of benefits the IRS has defined to date for
transition state 1 (TS1) and beyond, a related quantitative target for
a benefit if applicable, and the transition state in which IRS expects
to realize the benefit. According to IRS, defining which benefits will
be delivered in transition state 2 (TS2) is contingent upon a number
of decisions yet to be made and actual benefits may differ from those
currently expected depending on the outcome of these decisions.
Table 7: CADE 2 Benefits:
Benefit category: Taxpayer service benefits;
Benefit subcategory: Faster refunds and processing (four benefits);
Qualitative benefit: Increase in the percentage of refunds identified
for daily processing;
Quantitative target: Increase the percentage of refunds for daily
processing from 30 percent to 80 percent;
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Faster refunds and processing (four benefits);
Qualitative benefit: Reduce interest paid on refund amount owed to
taxpayer due to delays;
Quantitative target: Reduce interest paid on taxpayer refunds to
$38.00 per 1 million dollars (i.e., the total refund interest divided
by total refund dollars issued on original settlement);
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Faster refunds and processing (four benefits);
Qualitative benefit: Increase in daily updates for Web based
applications for individual taxpayers;
Quantitative target: Increase refunds identified for refund processing
to 100 percent;
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Faster refunds and processing (four benefits);
Qualitative benefit: Increase in transactions processed daily;
Quantitative target: Increase transactions processed daily to 100
percent compared to total IRS transactions identified for daily
processing;
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Faster payment posting (one benefit);
Qualitative benefit: Increase in taxpayer transactions for daily
processing will result in faster payment posting;
Quantitative target: Increase transactions posted daily to 80 percent
of payment transactions which include payments with extensions,
payments with returns and estimated payments;
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: More efficient account adjustment (two benefits);
Qualitative benefit: Increase in taxpayer transactions under a certain
threshold that will result in daily posted adjustments to accounts;
Quantitative target: Increase account adjustments processed daily to
80 percent compared to potential adjustment transactions;
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: More efficient account adjustment (two benefits);
Qualitative benefit: Reduce number of erroneous notices due to better
account information;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Reduction in manual refunds (one benefit);
Qualitative benefit: Improve IRS's ability to respond to a reduction
in the number of manual refund requests as a result of daily
processing;
Quantitative target: Not identified;
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Updated account information (three benefits);
Qualitative benefit: Reduce undelivered mail from quicker updates for
taxpayer name and address changes;
Quantitative target: Not identified.[A];
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Updated account information (three benefits);
Qualitative benefit: Provide earlier viewing of posted information
from daily updates to the Integrated Data Retrieval System (IDRS);
Quantitative target: Provide daily updates of transactions that are
viewable in IDRS within 48 hours (2 business days) after posting;
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Updated account information (three benefits);
Qualitative benefit: Reduce labor intensive research on internal and
external third party data;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit subcategory: Benefit category: Faster notices (one benefit);
Qualitative benefit: Distribute notices earlier than legacy timeframes
as a result of daily processing;
Quantitative target: Increase of notices posted daily by 25 percent
over CADE (2010) baseline;
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Systematic resolution (four benefits);
Qualitative benefit: Reduce merge processing time of taxpayer accounts
and data to support problem resolution;
Quantitative target: Reduce time combining taxpayer information in
CADE 2 by 50 percent;
Transition state: TS1.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Systematic resolution (four benefits);
Qualitative benefit: Improve refund time from better case selection
using additional analytical tools;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Systematic resolution (four benefits);
Qualitative benefit: Reduce process costs to IRS from reducing case
closure by hand and improve accuracy (decrease in transposition
errors, name control mismatch, etc.);
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Taxpayer service benefits;
Benefit subcategory: Systematic resolution (four benefits);
Qualitative benefit: Provide timely refund from better case selection;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Compliance benefits;
Benefit subcategory: Decreasing taxpayer overpayments (one benefit);
Qualitative benefit: Improve account accuracy by decreasing the
percentage of overpayments;
Quantitative target: Not identified;
Transition state: TS1.
Benefit category: Compliance benefits;
Benefit subcategory: Increasing taxpayer compliance (four benefits);
Qualitative benefit: Modify automated filters during first few weeks
of filing season based on analysis of tax returns filed early in the
filing season;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Compliance benefits;
Benefit subcategory: Increasing taxpayer compliance (four benefits);
Qualitative benefit: Freeze erroneous tax refund, eliminating
collection activity;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Compliance benefits;
Benefit subcategory: Increasing taxpayer compliance (four benefits);
Qualitative benefit: Implement a common module for penalty and
interest calculation with documented standard posting rules;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Compliance benefits;
Benefit subcategory: Increasing tax preparer compliance (two benefits);
Qualitative benefit: Modify automated filters during first few weeks
of filing season based on analysis of preparers of tax returns filed
early in the filing season;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Compliance benefits;
Benefit subcategory: Increasing tax preparer compliance (two benefits);
Qualitative benefit: Freeze erroneous tax refund, eliminating
collection activity;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Compliance benefits;
Benefit subcategory: Improved audit case selection (three benefits);
Qualitative benefit: Reduce no change rate, reduce taxpayer burden
associated with unnecessary audits of compliant taxpayers, and
increase yield per audit with more complete and timely data;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Compliance benefits;
Benefit subcategory: Improved audit case selection (three benefits);
Qualitative benefit: Apply advanced decision analytics earlier in
collection process after first notice;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Compliance benefits;
Benefit subcategory: Improved audit case selection (three benefits);
Qualitative benefit: Assign cases to optimal treatment strategies at
an earlier stage;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Compliance benefits;
Benefit subcategory: Improved audit case selection (three benefits);
Benefit subcategory: Apply advanced decision analytics earlier (after
first notice) that can lead to increased revenue collection by routing
audit cases earlier to the most effective treatment;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Other benefits;
Benefit subcategory: Reduce system maintenance costs (seven benefits);
Qualitative benefit: Improve data cleanliness and integrity;
Quantitative target: Improve distribution of data quality in the CADE
database with target of weighted average of the critical categories;
reduce the number of accounts with errors with target of 5 percent
reduction below 2011 baseline;
Transition state: TS1.
Benefit category: Other benefits;
Benefit subcategory: Reduce system maintenance costs (seven benefits);
Qualitative benefit: Improve data quality: data library where all data
is consistent, standardized, easy to understand and accurate;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Other benefits;
Benefit subcategory: Reduce system maintenance costs (seven benefits);
Qualitative benefit: Reduce time and effort to produce data extracts;
Quantitative target: Reduce number of requests for data extracts by 5
percent below 2011 baseline; reduce 50 percent below 701 extract
baseline reports from the Individual Master File tapes in a given
quarter;
Transition state: TS1.
Benefit category: Other benefits;
Benefit subcategory: Reduce system maintenance costs (seven benefits);
Qualitative benefit: Improve system maintainability;
Quantitative target: Increase in percentage of production systems
being monitored by automated monitoring and management systems with
target of 100 percent;
Transition state: TS1.
Benefit category: Other benefits;
Benefit subcategory: Reduce system maintenance costs (seven benefits);
Qualitative benefit: Reduce operational complexity in processing
accounts in one system;
Quantitative target: Increase in percentage of individual taxpayer
accounts that reside in relational database to a range of 96 percent
to 99 percent, and reduce costs to zero for CADE production support;
reduce hours spent to zero associated with CADE production support;
Transition state: TS1.
Benefit category: Other benefits;
Benefit subcategory: Reduce system maintenance costs (seven benefits);
Qualitative benefit: Reduce number of staff required to maintain
multiple versions of the same functionality through deployment of
common services;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Other benefits;
Benefit subcategory: Reduce system maintenance costs (seven benefits);
Qualitative benefit: Consolidate case management system;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Other benefits;
Benefit subcategory: Improved security (four benefits);
Qualitative benefit: Improve centralized control of the data;
Quantitative target: Reduce the number of databases at IRS that
contain extracts of data by 20 percent below 2011 baseline;
Transition state: TS1.
Benefit category: Other benefits;
Benefit subcategory: Improved security (four benefits);
Qualitative benefit: Improve security processing;
Quantitative target: Reduce number of issues identified in post
production security audits from the past quarter. Target: High: 0,
Moderate:5-10, Low: N/A;
Transition state: TS1.
Benefit category: Other benefits;
Benefit subcategory: Improved security (four benefits);
Qualitative benefit: Improve ability to track risk reduction to
taxpayer data provided by security requirements and controls;
Quantitative target: Not identified.[A];
Transition state: TS1.
Benefit category: Other benefits;
Benefit subcategory: Improved security (four benefits);
Qualitative benefit: Secure data in motion between systems; deploy
encryption solution for data in motion;
Quantitative target: Not identified;
Transition state: TS1.
Benefit category: Other benefits;
Benefit subcategory: Address financial material weakness (five
benefits);
Qualitative benefit: Improve compliance with Privacy and Disclosure
policy and protection of personally identifiable information;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Other benefits;
Benefit subcategory: Address financial material weakness (five
benefits);
Qualitative benefit: Address compliance with Federal Financial
Management System Requirements to improve financial integrity, and
reporting;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Other benefits;
Benefit subcategory: Address financial material weakness (five
benefits);
Qualitative benefit: Develop financial settlement application;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Other benefits;
Benefit subcategory: Address financial material weakness (five
benefits);
Qualitative benefit: Demonstrate incremental progress toward
supporting financial requirements for a subsidiary ledger;
Quantitative target: Not identified;
Transition state: TS2/target state.
Benefit category: Other benefits;
Benefit subcategory: Address financial material weakness (five
benefits);
Qualitative benefit: Demonstrate incremental progress toward
addressing material financial weaknesses;
Quantitative target: Not identified;
Transition state: TS2/target state.
Source: GAO analysis of IRS data.
[A] According to IRS officials, these benefits are not tangible and
are not quantifiable.
[End of table]
[End of section]
Appendix III: CADE 2 Program Risks:
The following (see table 8) are the IRS's reported CADE 2 program
risks and their planned mitigation strategies as of November 2010.
Table 8: IRS Reported CADE 2 Risks and Mitigation Strategies:
Risk: Undocumented individual taxpayer processing business
requirements;
IRS description: IRS's ability to rewrite Individual Master File
individual taxpayer processing code in a modernized language to
accurately reflect current business processing is dependent on
thorough and well-documented business requirements;
Mitigation: For TS2, the business has created and implemented a
harvesting plan to mitigate risks. The plan includes starting the
harvesting requirements early and increasing the business resources to
harvest business requirements.
Risk: TS1 contingency planning;
IRS description: If TS1 contingency planning and associated work
efforts are not addressed in a timely manner, the CADE 2 program may
not be able to exercise contingency options in the event that full TS1
is not ready for deployment in January 2012;
Mitigation: Presenting contingency options to governance boards and
updating contingency or deployment strategies/plans will be based on
decisions made by these boards.
Risk: Recovery time objectives--disaster recovery;
IRS description: Disaster recovery requirements for CADE 2 may not
align with current enterprise disaster recovery capabilities and
service level agreements, which may result in business impacts;
Mitigation: Modernization and Information Technology Services is
working with key stakeholders to develop an enterprisewide disaster
recovery strategy of which a piece of it is CADE 2. IRS will refocus
CADE 2 disaster recovery specific needs to be aligned with the
enterprisewide strategy.
Risk: Resource contention among IRS priorities;
IRS description: Filing season and other top IT investment priorities
may result in resource contention for key resources;
Mitigation: Establishing a process for monitoring and updating the
CADE 2 Staffing Strategy and Plan that includes understanding
government staffing needs in Delivery Partner organizations.
Risk: Database extract, transform, and load prototype 2;
IRS description: If the database extract, transform, and load
prototype results show that a component of the database process fails;
then, an alternative design solution will need to be recommended for
the timely completion of detailed design of the Database
Implementation project;
Mitigation: The database extract, transform, and load work breakdown
structure has included tasks to validate the resulting data and to
perform analysis for a recommended course of action after each
iteration is complete. The recommendation for go or no-go decision
points is made based on that analysis performed.
Risk: Availability of FY 2011 funding;
IRS description: It is critical that the CADE 2 program obtain
sufficient funds timely to successfully deploy the TS1 solution. If FY
2011 CADE 2 funding is not received within the first 3 months of FY
2011, the infrastructure for building out the production environment
and disaster recovery for TS1 development may be in jeopardy;
Mitigation: Making a deliberate decision to manage the risk through
financial operations control activities performed in conducting
business operations.
Risk: SharePoint performance;
IRS description: The CADE 2 program is using SharePoint as its primary
mechanism for ensuring information and work products are available to
key stakeholders. This increases the amount of time it takes to
access, review, and act on CADE 2 information stored in SharePoint. If
current SharePoint performance continues to be very slow, there will
be an impact on the program's ability to effectively access key data;
Mitigation: Enterprise Services is assisting the CADE 2 Program
Management Office (PMO) with moving all SharePoint content from the
existing SharePoint 2003 site to a SharePoint 2007 site. This will
improve the performance problems that generated this risk. The CADE 2
PMO is providing the necessary administrative access for Enterprise
Services so that the SharePoint content can be moved.
Risk: Extract, transform, and load separate platform problem-data
translation and data flow degradation;
IRS description: If there is not an alternative solution derived after
the engineering alternative analysis study is completed to improve
configuration of the extract, transform, and load on separate
platforms and to improve the performance data translation/movement of
data through transport stacks, the system data flow performance will
be degraded putting the data flow process at risk;
Mitigation: Enterprise services is leading the CADE 2 extract,
transform, and load Alternatives Analysis effort to address the risk
associated with possible data flow degradation. As a result of this
analysis, it has been recommended not to pursue any of the
alternatives to replace Electronic File Transfer Utility.
Risk: Database implementation challenges;
IRS description: The Database Implementation project faces many
challenges as the IRS moves toward establishing an authoritative
relational database. These challenges include a number of technical
hurdles and risks that were revealed as part of the recent extract,
transform, and load prototyping efforts;
Mitigation: Establishing a dedicated CADE Database Implementation
project team with end-to-end responsibility for delivery of the
database software component.
Risk: Requirements maturity;
IRS description: The successful implementation of the CADE 2 solution
is very dependent on a solid set of baselined requirements. If the
deficiencies identified in the current state of requirements are not
corrected in a timely manner, then, the delivery of CADE 2 TS1 may be
delayed;
Mitigation: Identifying the list of requirement deficiencies and
assigning a working group to validate corrective actions, assigning
ownership, and developing an implementation plan for completing each
correction action.
Source: IRS.
[End of table]
[End of section]
Appendix IV: Comments from the Internal Revenue Service:
Department Of The Treasury:
Internal Revenue Service:
Commissioner:
Washington, D.C. 20224:
March 10, 2011:
Mr. David Powner:
Director, Information Technology Management Issues:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Powner:
Thank you for the opportunity to comment on the draft report, Taxpayer
Account Strategy: IRS Should Finish Defining Benefits and Improve Cost
Estimates (GA0-11-168). Completion of the core taxpayer account
database is a top priority for the IRS, and I continue to appreciate
the valuable feedback that the GAO team has provided along the way.
The IRS's technology infrastructure runs the tax filing season every
year for approximately 140 million individual taxpayers, and manages
the distribution of over $300 billion in refunds. It is critically
important that we make appropriate investments to maintain this
important part of the United States' economic infrastructure.
I appreciate that your report recognizes the progress that the IRS has
made, and that we are following best practices on important
disciplines like risk management and cost estimation.
Even as we continue to make great progress, we can always do better.
We agree with your recommendations for improvement, and will provide a
detailed corrective action plan addressing each of the recommendations
with our response to the final report. In fact, we have already acted
on one of your recommendations related to the use of a sensitivity
analysis on our cost estimates. We completed the CADE 2 Revised Basis of
Estimate shortly after completion of your audit, which includes both a
sensitivity analysis and a summary of the differences between the
early Milestone 0 Basis of Estimate and the revised Basis of Estimate
(at Milestone 3).
We appreciate your continued support and guidance. If you have any
questions or would like to discuss our response, please contact
Terence V. Milholland, Chief Technology Officer, at (202) 622-6800.
Sincerely,
Signed by:
Douglas H. Shulman:
[End of section]
Appendix V: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
David A. Powner, (202) 512-9286, or pownerd@gao.gov James R. White,
(202) 512-9110, or whitej@gao.gov:
Staff Acknowledgments:
In addition to the individuals named above, Sabine R. Paul, Assistant
Director; Joanna Stamatiades, Assistant Director; Rebecca Eyler; Mary
D. Fike; Nancy Glover; Sairah R. Ijaz; Robert Kershaw; Paul B.
Middleton; John Ockay; and Jennifer Wong made key contributions to
this report.
[End of section]
Footnotes:
[1] A long history of continuing delays and design difficulties and
their impact on IRS's operations led us to designate IRS's systems
modernization as a high-risk area in 1995 and the program has since
remained high-risk. GAO, High-Risk Series: an Update, [hyperlink,
http://www.gao.gov/products/GAO-11-278] (Washington, D.C.: Feb. 16,
2011).
[2] At its inception, CADE was also intended to replace the Business
Master File, which contains taxpayer data about businesses, and the
Employee Master File, which contains data about employee retirement
plans. Over the years, due to implementation challenges, IRS scaled
back the project to only include the IMF.
[3] IRS, Customer Account Data Engine Database Implementation Project
Charter Transition State 1, Revision: 1.0 (Washington, D.C.: Mar. 22,
2010); Individual Master File (IMF) - Privacy Impact Assessment
(Washington, D.C. : June 7, 2007).
[4] Numbers are as of October 29, 2010, except for number of payments
processed, which are as of September 13, 2010, and payment dollar
totals, which are as of tax season 2009.
[5] IRS had five prototypes: (1) Java Viability: to test viability of
Java Common Operating Environment for CADE, including for posting and
settlement; (2) Database Extract, Transform, and Load: to evaluate
approaches to extract daily changes in the current IMF, transform to
database format, load onto the CADE 2 database; (3) Penalty and
Interest: to analyze existing computer code and identify changes to
allow common calculation; (4) Performance Modeling and Monitoring: to
create a performance model that is based on the workload that must be
processed in each CADE 2 major phase; and (5) Database Performance
Test: to assess whether the IMF daily updates can be extracted,
transformed, and loaded into the CADE 2 database within the required
daily time window.
[6] IRS proved that data can be extracted from the IMF, transformed,
and loaded into new CADE 2 database through the extract, transform,
and load prototype. However, the remaining question--whether the
database can be loaded within the daily time window IRS needs--and any
other open tasks are being rolled into the first phase of the CADE 2
Database integration project work.
[7] A prerequisite for Daily Processing is the implementation of a new
cycle definition that shifts the current weekly cycle to begin 1 day
sooner in the week. In August 2010, the CADE 2 Executive Steering
Committee approved the decision to accelerate the business and
corresponding technical cycle to this new cycle definition (also
referred to as Accelerated IMF Weekly) as a required component of TS1.
The new cycle will also accelerate data availability by a full week (7
days) to a key downstream IRS customer service system.
[8] IDRS is a mission critical system consisting of databases and
operating programs that support IRS employees working active tax cases
within each business function across IRS. This system manages data
that has been retrieved from the Tax Master Files allowing IRS
employees to take specific actions on taxpayer account issues, track
status, and post transaction updates back to the Tax Master Files. It
provides for systemic review of case status and notice issuance based
on case criteria, alleviating staffing needs and providing consistency
in case control.
[9] This will involve using data integration tools to parse the IMF
records, reformat the data to match the CADE 2 database table
structures, and establish the linkages and relationships required in
the CADE 2 database.
[10] Treasury Inspector General for Tax Administration, Reengineering
Individual Tax Return Processing Requires Effective Risk Management,
2010-20-001 (Washington, D.C.: Dec. 7, 2009).
[11] GAO, Business Systems Modernization: Internal Revenue Service's
Fiscal Year 2010 Expenditure Plan, [hyperlink,
http://www.gao.gov/products/GAO-10-539] (Washington, D.C.: May 10,
2010).
[12] Treasury Inspector General for Tax Administration, Prototype
Process Improvements Will Benefit Efforts to Modernize Taxpayer
Account Administration, 2011-20-001 (Washington, D.C.: Nov. 24, 2010).
[13] An enterprise architecture can be viewed as a blueprint for
organizational transformation and IT modernization. Generally
speaking, it consists of "snapshots" of the enterprise's current, or
"as-is," operational and technological environment and its target, or
"to-be," environment, and contains a capital investment road map for
transitioning from the current to the target environment. These
snapshots consist of "views," which are basically one or more
architecture products that provide conceptual, logical, or physical
representations of the enterprise. Further, these views or
representations are not static, but rather will evolve or change over
time, making enterprise architecture a "living document" (see GAO,
Organizational Transformation: A Framework for Assessing and Improving
Enterprise Architecture Management (Version 2.0) [hyperlink,
http://www.gao.gov/products/GAO-10-846G] (Washington, D.C.: August
2010).
[14] The enterprise life cycle is the approach used by IRS to manage
and implement business change through information systems initiatives.
The enterprise life cycle provides the direction, processes, tools,
and assets necessary to accomplish business change in a consistent and
repeatable manner. According to the enterprise life cycle, each
project must go through several phases including vision and strategy,
project initiation, preliminary design, detailed design, development,
and deployment.
[15] GAO, Tax Administration: IRS Needs to Further Refine Its Tax
Filing Season Performance Measures, [hyperlink,
http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: Nov. 22,
2002).
[16] Specific financial material weaknesses IRS expects to address are
weaknesses with the financial settlement application, common penalty
and interest module, and history of freezes. IRS also plans to create
an authoritative subsidiary ledger, comply with federal standards,
provide traceability of transactions to their source, and provide
traceability for financial report data to the detailed individual
transactions.
[17] Although the majority of information returns are due to taxpayers
by January 31 each year and to IRS by the end of February, IRS
generally does not begin using the data reported on information
returns for compliance purposes until several months later. It may not
be possible to require all information returns to be provided to IRS
at the same time they are provided to the taxpayer, given the
complexities of some forms. However, IRS officials have stated that
using third-party information earlier in the filing season could be
beneficial because it would allow IRS to match information on tax
returns to that data before it issues refunds.
[18] Because these costs were excluded in calculating the preliminary
estimate, the CADE 2 costs for TS1 and TS2 could potentially be higher
than the roughly $1.3 billion estimated.
[19] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009).
[20] According to the Estimation Program Office, based on the
estimation methodology used, the actual costs of the program for TS1
and TS2 could range from $975 million to $1.95 billion.
[21] Carnegie Mellon Software Engineering Institute, Capability
Maturity Model® Integration for Acquisition (CMMI-ACQ), Version 1.2
(November 2007).
[22] IRS, Customer Account Data Engine (CADE) 2 Risk and Issue
Management Process (May 31, 2010).
[23] The risk exposure score is a means of systematically evaluating
and prioritizing risks based upon the two dimensions of risk: its
likelihood of occurrence and its impact should it occur. Typically
these two dimensions are assigned a numeric value, and the two are
multiplied to obtain an exposure score. Plotting these scores on a two-
dimensional graph allows one to identify the priority to assign to the
risk based upon its location on the graph. Risks having a high score
are located in the region of the graph corresponding to a higher
probability of occurrence and a more serious impact.
[24] IRS, Customer Account Data Engine (CADE) 2 Program Management
Office Risk and Issue Management Plan (PMO-RIMP) (Oct. 15, 2010).
[25] IRS, CADE 2 Program TSI Contingency Strategy v. 1.1 (Nov. 1,
2010).
[26] [hyperlink, http://www.gao.gov/products/GAO-09-3SP].
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
