Business Systems Modernization
Internal Revenue Service's Fiscal Year 2011 Expenditure Plan Gao ID: GAO-12-26 October 6, 2011The Internal Revenue Service's (IRS) Business Systems Modernization (BSM) program is a multi-billion dollar, high-risk, highly complex effort that involves the development and delivery of a number of modernized systems that are intended to replace the agency's aging business and tax processing systems. As required, IRS submitted its fiscal year 2011 expenditure plan in May 2011 to the House and Senate appropriations committees, requesting approximately $352 million from the BSM account. In response to a mandate, GAO's objectives in reviewing the expenditure plan were to (1) determine whether it satisfies the applicable statutory conditions, (2) determine IRS's progress in implementing prior expenditure plan review recommendations, and (3) provide additional observations about the plan and the BSM program. To accomplish the objectives, GAO analyzed the plan, reviewed related documentation, and interviewed IRS officials.
IRS's expenditure plan satisfies each of the six applicable statutory conditions, which include meeting the Office of Management and Budget's (OMB) capital planning and investment control review requirements, and complying with federal systems acquisition requirements and management practices (as encompassed by the Software Engineering Institute's Capability Maturity Model Integration). IRS has addressed four of GAO's five outstanding recommendations from prior expenditure plan reviews. For example, IRS has developed high-level plans for the second phase of its Customer Account Data Engine (CADE) 2 program, IRS's new effort to replace its legacy systems for storing, managing, and accessing individual taxpayer accounts. However, steps remain to fully implement the recommendation to ensure that expenditure plans include a quantitative measure of progress in meeting project scope expectations. IRS expects to include this metric in its fiscal year 2012 expenditure plan. GAO has the following four observations about the expenditure plan and the BSM program: (1) GAO's analysis of reported project costs and completion dates shows that 11 of the 12 project milestones planned for completion between January 2010 and May 2011 were completed early, under budget, or within 10 percent of cost and schedule estimates, and 1 milestone was completed on schedule but significantly over cost. Specifically, a milestone for the Modernized e-File project was completed more than 70 percent over estimated costs. According to IRS, several factors contributed to the overrun, including the need to implement unplanned requirements for disaster recovery. (2) For fiscal year 2011 IRS requested about $174 million, or about 49 percent of the total BSM request for level of effort (LOE) work--that is, tasks of a general or supportive nature, such as program management, which do not result in clear products. According to best practices, if more than 15 percent of a program's budget is classified as LOE, this amount should be scrutinized. One factor contributing to the large percentage is that IRS's definition of LOE differs from the commonly accepted definition in that it includes some work that results in clear products. By categorizing such tasks as LOE in the expenditure plan, IRS is presenting an inaccurate picture of the work it is performing and the manner in which it is being managed, and providing Congress with less insight into its performance in implementing the BSM program. (3) CADE 2 risk management and preliminary cost estimating processes are generally consistent with best practices; however, IRS will be challenged in completing one of the two key projects for the first phase by January 2012, as planned. According to IRS officials, IRS is still on track for delivering this project by January 2012 as planned. However, the agency is considering phasing its implementation to reduce the impact on filing season operations. (4) Due, in part, to the fact that IRS has not yet fully implemented key components of its comprehensive information security program, IRS continues to have information security weaknesses. Although not all these weaknesses pertain to BSM specifically, IRS's information security program encompasses all agency systems, and therefore the program weaknesses affect the modernization environment. GAO's recommendations include reclassifying activities with discrete work and accordingly reporting on associated performance data in future expenditure plans, and consistent with best practices, scrutinizing any remaining LOE work exceeding 15 percent of the program's budget. In comments on a draft of this report, IRS noted disagreement over certain aspects of GAO's views on LOE work but stated that it will collaborate with GAO to ensure that this work is clearly expressed in the expenditure plan.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: David A. Powner Team: Government Accountability Office: Information Technology Phone: (202) 512-9286GAO-12-26, Business Systems Modernization: Internal Revenue Service's Fiscal Year 2011 Expenditure Plan
This is the accessible text file for GAO report number GAO-12-26
entitled 'Business Systems Modernization: Internal Revenue Service's
Fiscal Year 2011 Expenditure Plan' which was released on October 6,
2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Committees:
October 2011:
Business Systems Modernization:
Internal Revenue Service's Fiscal Year 2011 Expenditure Plan:
GAO-12-26:
GAO Highlights:
Highlights of GAO-12-26, a report to congressional committees.
Why GAO Did This Study:
The Internal Revenue Service‘s (IRS) Business Systems Modernization
(BSM) program is a multi-billion dollar, high-risk, highly complex
effort that involves the development and delivery of a number of
modernized systems that are intended to replace the agency‘s aging
business and tax processing systems. As required, IRS submitted its
fiscal year 2011 expenditure plan in May 2011 to the House and Senate
appropriations committees, requesting approximately $352 million from
the BSM account.
In response to a mandate, GAO‘s objectives in reviewing the
expenditure plan were to (1) determine whether it satisfies the
applicable statutory conditions, (2) determine IRS‘s progress in
implementing prior expenditure plan review recommendations, and (3)
provide additional observations about the plan and the BSM program. To
accomplish the objectives, GAO analyzed the plan, reviewed related
documentation, and interviewed IRS officials.
What GAO Found:
IRS‘s expenditure plan satisfies each of the six applicable statutory
conditions, which include meeting the Office of Management and Budget‘
s (OMB) capital planning and investment control review requirements,
and complying with federal systems acquisition requirements and
management practices (as encompassed by the Software Engineering
Institute‘s Capability Maturity Model Integration).
IRS has addressed four of GAO‘s five outstanding recommendations from
prior expenditure plan reviews. For example, IRS has developed high-
level plans for the second phase of its Customer Account Data Engine
(CADE) 2 program, IRS‘s new effort to replace its legacy systems for
storing, managing, and accessing individual taxpayer accounts.
However, steps remain to fully implement the recommendation to ensure
that expenditure plans include a quantitative measure of progress in
meeting project scope expectations. IRS expects to include this metric
in its fiscal year 2012 expenditure plan.
GAO has the following four observations about the expenditure plan and
the BSM program:
* GAO‘s analysis of reported project costs and completion dates shows
that 11 of the 12 project milestones planned for completion between
January 2010 and May 2011 were completed early, under budget, or
within 10 percent of cost and schedule estimates, and 1 milestone was
completed on schedule but significantly over cost. Specifically, a
milestone for the Modernized e-File project was completed more than 70
percent over estimated costs. According to IRS, several factors
contributed to the overrun, including the need to implement unplanned
requirements for disaster recovery.
* For fiscal year 2011 IRS requested about $174 million, or about 49
percent of the total BSM request for level of effort (LOE) work”-that
is, tasks of a general or supportive nature, such as program
management, which do not result in clear products. According to best
practices, if more than 15 percent of a program's budget is classified
as LOE, this amount should be scrutinized. One factor contributing to
the large percentage is that IRS‘s definition of LOE differs from the
commonly accepted definition in that it includes some work that
results in clear products. By categorizing such tasks as LOE in the
expenditure plan, IRS is (1) presenting an inaccurate picture of the
work it is performing and the manner in which it is being managed, and
(2) providing Congress with less insight into its performance in
implementing the BSM program.
* CADE 2 risk management and preliminary cost estimating processes are
generally consistent with best practices; however, IRS will be
challenged in completing one of the two key projects for the first
phase by January 2012, as planned. According to IRS officials, IRS is
still on track for delivering this project by January 2012 as planned.
However, the agency is considering phasing its implementation to
reduce the impact on filing season operations.
* Due, in part, to the fact that IRS has not yet fully implemented key
components of its comprehensive information security program, IRS
continues to have information security weaknesses. Although not all
these weaknesses pertain to BSM specifically, IRS‘s information
security program encompasses all agency systems, and therefore the
program weaknesses affect the modernization environment.
What GAO Recommends:
GAO‘s recommendations include reclassifying activities with discrete
work and accordingly reporting on associated performance data in
future expenditure plans, and consistent with best practices,
scrutinizing any remaining LOE work exceeding 15 percent of the program‘
s budget. In comments on a draft of this report, IRS noted
disagreement over certain aspects of GAO's views on LOE work but
stated that it will collaborate with GAO to ensure that this work is
clearly expressed in the expenditure plan.
View [hyperlink, http://www.gao.gov/products/GAO-12-26]. For more
information, contact David A. Powner at (202) 512-9286 or
pownerd@gao.gov.
[End of section]
Contents:
Letter:
Conclusions:
Recommendations for Executive Action:
Agency Comments:
Appendix I: Briefing Slides from the July 20, 2011, Briefing to the
Senate and House Appropriations Subcommittee Staffs:
Appendix II: Comments from the Internal Revenue Service:
Appendix III: GAO Contact and Staff Acknowledgments:
Abbreviations:
AMS: Accounts Management Services:
BSM: Business Systems Modernization:
CADE: Customer Account Data Engine:
CIO: Chief Information Officer:
CMMI: Capability Maturity Model Integration:
EA: enterprise architecture:
IRS: Internal Revenue Service:
IT: information technology:
LOE: level of effort:
MeF: Modernized e-File:
MITS: Modernization and Information Technology Services:
OMB: Office of Management and Budget:
SEI: Software Engineering Institute:
TIGTA: Treasury Inspector General for Tax Administration:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
October 6, 2011:
The Honorable Richard J. Durbin:
Chairman:
The Honorable Jerry Moran:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
United States Senate:
The Honorable Jo Ann Emerson:
Chairwoman:
The Honorable José E. Serrano:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
House of Representatives:
As required by the Consolidated Appropriations Act, 2010, the Internal
Revenue Service (IRS) submitted its fiscal year 2011 expenditure plan
in May 2011[Footnote 1] to the House and Senate appropriations
committees, requesting approximately $352 million from the Business
Systems Modernization (BSM) account, which funds IRS's efforts to
modernize its business and tax processing systems.[Footnote 2]The
statute also requires GAO to review the expenditure plan. Our
objectives in reviewing the plan were to (1) determine whether the
plan satisfies the applicable statutory conditions,[Footnote 3] (2)
determine IRS's progress in implementing outstanding recommendations
from our prior year expenditure plan reviews, and (3) provide any
other observations about the plan and IRS's BSM program.
This report transmits the information we provided to congressional
appropriations subcommittee staffs during our July 20, 2011, briefing
and provides the recommendations that we made to the Commissioner of
Internal Revenue. The full briefing materials, including our scope and
methodology, are included as appendix I.
We conducted this performance audit from May 2011 to July 2011 in
Washington, D.C., in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objectives.
In summary, we made the following major points:
* IRS's fiscal year 2011 plan satisfies each of the six statutory
conditions. These conditions include meeting OMB's capital planning
and investment control review requirements, and complying with federal
systems acquisition requirements and management practices.[Footnote 4]
* IRS has implemented four of our five outstanding recommendations
from prior expenditure plan reviews. For example, the agency developed
high-level plans for the second phase of the Customer Account Data
Engine (CADE) 2, IRS's new effort to replace its legacy systems for
storing, managing, and accessing individual taxpayer accounts.
However, steps remain to implement our recommendation to ensure that
expenditure plans include a quantitative measure of progress in
meeting project scope expectations. IRS expects to include this metric
in its fiscal year 2012 expenditure plan.
* Of 12 project milestones[Footnote 5] scheduled for completion
between January 2010 and May 2011, 11 were completed within 10 percent
of cost and schedule estimates, under budget, or early, and 1 was
completed on schedule but significantly over cost. For example, both
the logical and physical design of CADE 2 were completed as budgeted
and earlier than planned. However, a release of Modernized e-File,
IRS's system intended to provide a single standard for filing
electronic tax returns, was more than 70 percent over estimated costs
at the completion of its deployment milestone. According to IRS,
several factors contributed to the overrun, including the need to
implement unplanned requirements for disaster recovery.
* A large percentage of requested BSM funding is categorized as level
of effort (LOE) work. For fiscal year 2011 IRS requested about $174
million, or about 49 percent of the total BSM request, for LOE work--
that is, tasks of a general or supportive nature, such as program
management, which do not result in clear products. According to best
practices, if more than 15 percent of a program's budget is classified
as LOE, this amount should be scrutinized. One factor contributing to
the large percentage of requested LOE funding in the BSM expenditure
plan is that IRS's definition of LOE differs from the commonly
accepted definition in that it includes some work that results in
clear work products. By categorizing such tasks as LOE in the
expenditure plan, IRS is (1) presenting an inaccurate picture of the
work it is performing and the manner in which it is being managed, and
(2) providing Congress with less insight into its performance in
implementing the BSM program.
* CADE 2 risk management and preliminary cost estimating processes are
generally consistent with best practices; however, IRS will be
challenged in completing one of the two key projects by January 2012,
as planned. In addition, IRS has developed a baseline estimate for the
first phase of the CADE 2 program. We recently reported that IRS
processes for managing risk and estimating preliminary costs for the
CADE 2 program were generally consistent with best practices and made
recommendations for improving the credibility of the cost
estimate.[Footnote 6] We also reported that the schedule for
delivering the first phase, consisting of the Daily Processing and
Database Implementation projects, was ambitious. IRS officials
acknowledged this and took actions to address this schedule challenge.
A recent independent assessment of the first phase of CADE 2 confirmed
that IRS would be challenged in delivering the Database Implementation
project on schedule. According to the Deputy Chief Information Officer
for Strategy and Modernization and the Program Director for CADE 2,
IRS is still on track for delivering the database by January 2012 as
planned. However, the agency is considering phasing its implementation
to reduce the impact on filing season operations. IRS also updated its
cost estimate for the first phase of the CADE 2 program, estimating
that the phase will cost $241 million to complete.[Footnote 7]
According to the Deputy Chief Information Officer for Strategy and
Modernization, this estimate will serve as the baseline for the phase.
* Information security weaknesses continue to affect IRS's
modernization environment. In March 2011,[Footnote 8] we reported that
due, in part, to the fact that IRS has not yet fully implemented key
components of its comprehensive information security program, 65 out
of the 88 previously reported information security weaknesses remained
unresolved and new weaknesses were identified. Although not all these
weaknesses pertain to BSM specifically, IRS's information security
program encompasses all agency systems, and therefore the program
weaknesses affect the modernization environment. We recommended that
IRS take eight actions to fully implement key components of its
comprehensive information security program. In June 2011, IRS provided
us with the status of actions taken and activities planned to address
these recommendations. We have not yet determined their effectiveness
in addressing our recommendations.
Conclusions:
IRS's fiscal year 2011 expenditure plan satisfies all six statutory
conditions. Further, IRS has made progress in addressing most of the
outstanding recommendations from our prior expenditure plan reviews.
However, IRS is still working to include a quantitative measure of
progress in meeting scope expectations in the plan, a recommendation
we made over 4 years ago. Until IRS fully implements this
recommendation, Congress may not have the information it needs to
effectively assess the implementation of the BSM effort.
During fiscal year 2010, IRS successfully completed 11 out of 12
project milestones within 10 percent of cost and schedule, under
budget, or early. However, one milestone was completed more than 70
percent over cost due to unplanned requirements. Further, IRS has
categorized a large amount--about 50 percent--of the funding requested
in the expenditure plan as level of effort, including for work that
does not appear to be appropriately characterized as such. By not
ensuring that all funding is appropriately categorized, IRS is
presenting an inaccurate picture of the work it is performing and the
manner in which it is being managed and providing Congress with less
insight into its performance in implementing the BSM program.
IRS has also taken key steps to ensure that its CADE 2 program is
being successfully managed. However, the program's schedule is
ambitious, and this has been corroborated by a recent independent
assessment. IRS officials have stated they are still on track for
completing planned work on schedule but are taking steps to reduce
risks.
Finally, weaknesses in information security continue to affect IRS's
modernization environment, largely because the agency has not fully
implemented key components of a comprehensive information security
program. Both we and the Treasury Inspector General for Tax
Administration have reported on these weaknesses, and full
implementation of outstanding recommendations will be needed for IRS
to ensure the security of its systems and the information they contain.
Recommendations for Executive Action:
To more accurately represent the work IRS is performing and provide
Congress greater insight into the agency's performance in implementing
BSM, we recommend that the Commissioner of Internal Revenue direct the
Chief Technology Officer to take the following three actions:
1. Modify IRS's definition of level of effort work to be consistent
with the commonly accepted definition.
2. Reclassify activities involving discrete work and accordingly
report on associated performance data in future expenditure plans.
3. Consistent with best practices, scrutinize any remaining LOE work
exceeding 15 percent of the program's budget to look for opportunities
to reduce the amount.
Agency Comments and Our Evaluation:
IRS's Commissioner provided written comments on a draft of this report
(reprinted in app. II). He stated that he appreciated that the report
recognized that the expenditure plan satisfied each of the six
applicable statutory conditions, and that IRS valued GAO's feedback,
having implemented four of the five outstanding recommendations. He
further stated that IRS plans to implement the final recommendation--
including a quantitative measure of progress in meeting scope--in the
fiscal year 2012 expenditure plan.
The Commissioner also stated that while IRS staff had expressed
disagreement over certain aspects of our feedback on cost estimation,
the agency would continue to engage in constructive dialog to ensure
that the level of effort it puts into specific deliverables is clearly
expressed in the expenditure plan. We look forward to working with IRS
as it addresses the representation of LOE within the expenditure plan.
Regarding information security, the Commissioner stated that IRS has
made substantial progress in the resolution of identified concerns
over the past year.
We are sending copies of this report to the Chairs and Ranking Members
of the Senate and House committees and subcommittees that have
appropriations, authorization, and oversight responsibilities for IRS.
We are also sending copies to the Commissioner of Internal Revenue,
the Secretary of the Treasury, the Chairman of the IRS Oversight
Board, and the Director of OMB. In addition, the report will be
available at no charge on the GAO website at [hyperlink,
http://www.gao.gov].
Should you and your offices have questions on matters discussed in
this report, please contact me at (202) 512-9286 or at
pownerd@gao.gov. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. GAO staff who made key contributions to this report are listed
in appendix III.
Signed by:
David. A. Powner:
Director, Information Technology Management Issues:
[End of section]
Appendix I: Briefing Slides from the July 20, 2011, Briefing to the
Senate and House Appropriations Subcommittee Staffs:
Review of IRS's Fiscal Year 2011 Business Systems Modernization
Expenditure Plan:
Briefing for Staff Members of the:
Subcommittee on Financial Services and General Government, Committee
on Appropriations, U.S. Senate:
and the:
Subcommittee on Financial Services and General Government, Committee
on Appropriations, House of Representatives:
July 20, 2011:
Contents:
Introduction and Objectives:
Scope and Methodology:
Results in Brief:
Background:
Results:
Conclusions:
Recommendations for Executive Action:
Appendixes:
I--Description of Business Systems Modernization Projects and Program-
Level Initiatives:
II--Additional Detail on IRS's Fiscal Year 2011 Business Systems
Modernization Expenditure Plan:
III--IRS Reported Cost and Schedule Variance for Project Milestones:
[End of section]
Introduction and Objectives:
The Internal Revenue Service's (IRS) Business Systems Modernization
(BSM) program is a multi-billion-dollar, high-risk, highly complex
effort that involves the development and delivery of a number of
modernized tax administration and internal management systems, as well
as core infrastructure projects, that are intended to replace the
agency's aging business and tax processing systems and provide
improved and expanded service to taxpayers and internal business
efficiencies for IRS. IRS's history of modernization spans several
decades, and BSM, initiated in fiscal year 1999, is IRS's most recent
modernization effort.
In 2008, prompted by several challenges confronting its Customer
Account Data Engine (CADE) program--intended to replace legacy systems
for storing, managing, and accessing taxpayer accounts[Footnote 9]--
the IRS Commissioner initiated a study of IRS's information technology
(IT) systems modernization efforts from an overall portfolio
perspective, which included operations of existing systems, new system
developments, and information security. The results of the study led
to the decision to refocus modernization efforts on the completion of
the modernized taxpayer account database. IRS expects the resulting
strategy, generally referred to as CADE 2, to deliver key benefits
beginning in filing season 2012. Specifically, through CADE 2, IRS
expects to (1) accelerate delivery of a relational taxpayer account
database from at least 7 years under the initial approach to
approximately 2 years, and (2) more quickly move to a single tax
processing environment (instead of the two environments that currently
exist). IRS has reported that CADE 2 is central to its vision for
ongoing improvements to its tax administration systems, and it is
expected that CADE 2 will result in faster refunds, improved customer
service, elimination of notices based on out-of-date information,
faster resolution of taxpayer account issues, and faster updates for
Web-based tools and services for individual taxpayers. In addition to
CADE 2, the fiscal year 2011 BSM request includes funding for CADE,
Modernized e-File (MeF)--an electronic tax return filing system--and
funding for IT infrastructure, architecture and integration, and
program management activities.
The Full-Year Continuing Appropriations Act, 2011,[Footnote 10]
provides appropriations for IRS for fiscal year 2011, including for
BSM. The act prohibits IRS from obligating funds (excluding labor
costs) for the BSM program until IRS submits a modernization
expenditure plan to the congressional appropriations committees.
[Footnote 11]
This plan must:
* meet the capital planning and investment control review requirements
established by the Office of Management and Budget (OMB);
* comply with IRS's enterprise architecture (EA);[Footnote 12]
* conform with IRS's enterprise life cycle methodology;[Footnote 13]
* comply with federal acquisition rules, requirements, guidelines, and
systems acquisition management practices;
* be approved by IRS, the Department of the Treasury, and OMB; and:
* be reviewed by GAO.
Since mid-1999, IRS has submitted a series of expenditure plans for
BSM appropriations. To date, IRS has received about $3.1 billion for
the BSM effort.
On May 10, 2011, IRS submitted its fiscal year 2011 expenditure plan
to the Senate and House of Representatives Committees on
Appropriations, seeking release of approximately $352 million from the
BSM account.[Footnote 14]
As agreed with IRS's appropriations subcommittees, our objectives were
to:
* determine whether IRS's fiscal year 2011 expenditure plan satisfies
the applicable statutory conditions;
* provide an update on IRS's progress in implementing our prior
expenditure plan review recommendations; and:
* provide any other observations about the expenditure plan and IRS's
BSM program.
[End of section]
Scope and Methodology:
To accomplish our objectives, we:
* reviewed the fiscal year 2011 expenditure plan submitted by IRS in
May 2011;
* analyzed the plan for compliance with the applicable statutory
conditions;
* interviewed IRS program and project management officials to
corroborate our understanding of the plan and other BSM activities;
* analyzed documentation on IRS's recent efforts to implement the
outstanding recommendations from our prior expenditure plan reviews;
* reviewed and analyzed modernization program reviews and project
management briefings and related documentation to assess program and
project status and associated issues and risks;
* we did not verify the cost and schedule data for the project
milestones we tracked, but we corroborated the data reported in the
expenditure plan with data found in program governance documents,
including project control reports, meeting minutes, and milestone exit
decision memos;
* reviewed program management reports to assess the progress IRS has
made in completing actions and implementing program management
improvements; and:
* reviewed related reports by the Treasury Inspector General for Tax
Administration (TIGTA).
From these efforts, we believe the information we obtained is
sufficiently reliable for this report.
We conducted this performance audit from May 2011 to July 2011 in
Washington, D.C., in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objectives.
[End of section]
Results in Brief:
IRS's fiscal year 2011 expenditure plan satisfies all six statutory
conditions.
IRS has implemented four of our five outstanding recommendations from
prior expenditure plan reviews. For example, the agency developed high-
level plans for the second phase of CADE 2. However, steps remain to
implement our recommendation to ensure that expenditure plans include
a quantitative measure of progress in meeting project scope
expectations. IRS expects to include this metric in its fiscal year
2012 expenditure plan.
We have four observations related to the BSM program and fiscal year
2011 expenditure plan:
* Of 12 project milestones[Footnote 15] scheduled for completion
between January 2010 and May 2011, 11 were completed within 10 percent
of cost and schedule estimates, under budget, or early, and 1 was
completed on schedule but significantly over cost. For example, both
the logical and physical design of CADE 2 were completed as budgeted
and earlier than planned. However, a release of MeF was more than 70
percent over estimated costs at the completion of its deployment
milestone. According to IRS, several factors contributed to the
overrun, including the need to implement unplanned requirements for
disaster recovery.
* A large percentage of requested BSM funding is categorized as level
of effort work. IRS is requesting about $174 million, or 49 percent of
the total BSM request, for level of effort (LOE) work--that is, work
consisting of tasks of a general or supportive nature, such as program
management, which does not result in clear work products. We have
previously reported that if more than 15 percent of a program's budget
is classified as level of effort, best practices call for this amount
to be scrutinized. One factor contributing to the large percentage of
funding for LOE work requested in the BSM expenditure plan is that
IRS's definition of LOE differs from the commonly accepted definition
in that it includes discrete work, i.e., work that results in clear
work products. By categorizing discrete work as LOE in the expenditure
plan, IRS is (1) presenting an inaccurate picture of the work it is
performing and the manner in which it is being managed, and (2)
providing Congress with less insight into its performance in
implementing the BSM program.
* CADE 2 risk management and preliminary cost estimating processes are
generally consistent with best practices; however, IRS will be
challenged in completing one of two key projects by January 2012 as
planned. In addition, IRS has developed a baseline estimate for the
first phase of the CADE 2 program. We recently reported that IRS
processes for managing risk and estimating preliminary costs for the
CADE 2 program were generally consistent with best practices and made
recommendations for improving the credibility of the cost estimate.
[Footnote 16] We also reported that the schedule for delivering the
first phase, consisting of the Daily Processing and Database
Implementation projects, was ambitious. IRS officials acknowledged
this and took actions to address this schedule challenge. A recent
independent assessment of the first phase of CADE 2 confirmed that IRS
would be challenged in delivering the Database Implementation project
on schedule. According to the Deputy Chief Information Officer for
Strategy and Modernization and the Program Director for CADE 2, IRS is
still on track for delivering the database by January 2012 as planned.
However, the agency is considering phasing its implementation to
reduce the impact on filing season operations. IRS also updated its
cost estimate for the first phase of the CADE 2 program, estimating it
will cost $241 million to complete. According to the Deputy Chief
Information Officer for Strategy and Modernization, this estimate will
serve as the baseline for the phase.
* Information security weaknesses continue to affect IRS's
modernization environment. In March 2011,[Footnote 17] we reported
that the 65 out of the 88 previously reported information security
weaknesses which remained unresolved, and new weaknesses identified
were due, in part, to the fact that IRS has not yet fully implemented
key components of its comprehensive information security program.
Although not all these weaknesses pertain to BSM specifically, IRS's
information security program encompasses all agency systems, and
therefore the program weaknesses affect the modernization environment.
We recommended that IRS take eight actions to fully implement key
components of its comprehensive information security program. In June
2011, IRS provided us with the status of actions taken and activities
planned to address these recommendations. We have not yet determined
their effectiveness in addressing our recommendations. In a separate
review, the Treasury Inspector General for Tax Administration also
found that IRS prematurely reported resolution of six vulnerabilities
related to the current CADE system before effective corrective actions
were taken.
To more accurately represent the work IRS is performing and provide
Congress greater insight into the agency's performance in implementing
BSM, we recommend that the Commissioner of Internal Revenue direct the
Chief Technology Officer take the following three actions:
(1) modify IRS's definition of level of effort work to be consistent
with the commonly accepted definition;
(2) reclassify activities involving discrete work and accordingly
report on associated performance data in future expenditure plans; and:
(3) consistent with best practices, scrutinize any remaining LOE work
exceeding 15 percent of the program's budget to look for opportunities
to reduce the amount.
In e-mail comments on a draft of these briefing slides, IRS officials,
including the Deputy Chief Information Officer for Strategy and
Modernization, agreed with our recommendation to accurately represent
the work IRS is performing and provide Congress with ongoing insight
into the agency's performance in implementing BSM. However, the agency
stated it did not agree with the actions we specified it take to
implement our recommendation: IRS did not specifically address reasons
for this disagreement. Nevertheless, IRS stated it would like to work
with us on how to best implement our recommendation to ensure an
adequate level of insight into implementation of the BSM program.
[End of section]
Background:
Since 1999, we have reviewed and reported on numerous BSM expenditure
plans. In particular, we have reported on program management
capabilities and controls that are critical to the effective
management of the BSM program. They include:
* cost and schedule estimates: IRS did not have effective procedures
for validating contractor-developed cost and schedule estimates;
[Footnote 18]
* requirements development and management: IRS did not have adequate
policies and procedures in place to guide its system modernization
projects in developing and managing requirements;[Footnote 19]
* post-implementation reviews: post-implementation reviews conducted
were incomplete and did not follow IRS procedures;[Footnote 20]
* quantitative measures of progress in scope: IRS expenditure plans
did not have a quantitative measure of progress in meeting project
scope expectations;[Footnote 21] and:
* conditional milestone exits: IRS did not have documented procedures
for determining when projects were allowed to proceed to the next
milestone with outstanding issues remaining to be addressed (i.e.,
conditional milestone exits).[Footnote 22]
Over the years, we have made recommendations aimed at strengthening
IRS's program management controls and capabilities, and IRS has worked
to address them.
IRS's Modernization and Information Technology Services (MITS)
organization has primary responsibility for managing and delivering
the BSM program. It is headed by a Chief Technology Officer[Footnote
23] who is supported by, among others, the Deputy Chief Information
Officer (CIO) for Strategy/Modernization and the Deputy CIO for
Operations. MITS is comprised of nine Associate CIO-level
organizations, including the following, which are involved in the BSM
program.
* Applications Development, which is responsible for building,
testing, delivering and maintaining integrated information
applications systems (i.e., software solutions) to support modernized
systems and the production environment;
* Enterprise Services, which is responsible defining the current and
target EAs and developing a transition strategy to move the agency
toward the target environment;
* Strategy and Planning, which is responsible for collaborating with
IT leadership to provide policy, direction, and administration of
essential programs, including strategy and capital planning, strategic
planning and performance measurement, financial management services,
vendor and contract management, requirements and demand management,
and risk management; and:
* Modernization Program Management, which is responsible for
overseeing the implementation of CADE 2.
IRS's fiscal year 2011 expenditure plan describes the agency's efforts
to develop modernized systems and supporting infrastructure.[Footnote
24] They include:
* continuing ongoing program-level initiatives (e.g., architecture and
integration and program management) and core infrastructure projects
(e.g., infrastructure shared services), and:
* continuing segments of three tax administration projects.
Key tax administration projects include:
* MeF, which is to provide a single standard filing structure for all
types of IRS tax returns;
* Current CADE,[Footnote 25] which was intended to provide the
modernized database foundation to replace the existing Individual
Master File processing system. In fiscal year 2011, the system is
expected to continue with daily processing of tax returns for
approximately 40 million taxpayers and implementing new tax law
changes; and:
* CADE 2, which is intended to provide the modernized database
foundation and leverage applications, database structures, files, and
business logic from both current CADE and Individual Master File
[Footnote 26] to provide a daily processing capability for moving all
individual taxpayer accounts to a faster refund cycle and address
IRS's long-term modernization plans and goals.
Details on these and other BSM projects and program-level initiatives
identified in the fiscal year 2011 plan are provided in appendix I.
Table 1 below shows a financial summary of the plan. Further details
on the plan are provided in appendix II.
Table 1: Summary of IRS's Fiscal Year 2011 BSM Expenditure Plan:
Tax administration projects:
MeF: $39,100,000.
Current CADE[A]: $28,000,000.
CADE 2: $136,100,000.
Subtotal--tax administration projects, Core infrastructure projects:
$203,200,000.
Development, Integration, and Testing Environments: $14,500,000.
Infrastructure Shared Services: $17,500,000.
Subtotal--core infrastructure projects, Architecture, integration, and
management: $32,000,000.
Architecture and integration: $11,200,000.
Program governance and control: $1,650,000.
Requirements management: $2,670,000.
Management processes: $3,250,000.
Federally funded research and development center: $6,050,000.
Project management: $2,735,000.
Subtotal--architecture, integration, and management: $27,555,000.
Management reserve: $8,925,000.
BSM capital total: $271,680,000.
BSM labor[B]: $80,689,000.
Maintaining current levels: $0.
Total; $352,369,000.
Source: GAO analysis of IRS data.
[A] IRS has decided it will not use Current CADE for the January 2012
filing season. According to officials, they plan to seek internal
approval to reprogram the $13 million of $28 million that was
requested for a new release of the program to CADE 2. According to IRS
guidance, once internal approval is obtained to reprogram BSM funds,
the agency is required to issue a congressional notification of the
change, which is reviewed and approved by Treasury and OMB prior to
issuance.
[B] The BSM labor request is a 75 percent increase over the FY 2010
request ($46 million). IRS officials attributed the increase to CADE 2
labor needs.
[End of table]
[End of section]
Results: Statutory Conditions:
IRS‘s Fiscal Year 2011 BSM Expenditure Plan Satisfies the Six
Statutory Conditions:
As in years past, in order to receive fiscal year 2011 funding, IRS
was required to submit an expenditure plan that satisfied the
following six statutory conditions. We found that the plan satisfied
all six.
Table 2: Status of Fiscal Year 2011 Expenditure Plan Provisions for
Satisfying Statutory Conditions[Footnote 27]
Statutory condition: 1. Meets OMB capital planning and investment
control review requirements;
Satisfied.
Statutory condition: 2. Complies with IRS's EA;
Satisfied.
Statutory condition: 3. Conforms with IRS's enterprise life cycle
methodology;
Satisfied.
Statutory condition: 4. Complies with the acquisition rules,
requirements, guidelines, and systems acquisition management practices
used by the federal government;
Satisfied.
Statutory condition: 5. Approved by IRS, the Department of the
Treasury, and OMB;
Satisfied.
Statutory condition: 6. Reviewed by GAO;
Satisfied.
Source: GAO analysis of IRS data.
[End of table]
1. IRS's expenditure plan meets OMB capital planning and investment
control review requirements:
OMB requires that agencies establish policy for planning, budgeting,
acquisition, and management of federal capital assets. More
specifically, it calls for agencies to develop capital planning and
investment control review processes that help ensure that projects are
being implemented at an acceptable cost and within reasonable and
expected time frames, and that they are contributing to observable
improvements in mission performance. The BSM expenditure plan met this
condition. Specifically, as noted in the plan, IRS has established a
structured governance and decision-making process that includes
various levels of governance bodies to manage its projects under a
standardized approach.
We recently performed an assessment of this process against the best
practices identified in our Information Technology Investment
Management Framework[Footnote 28] and found it to be generally
consistent with best practices and OMB capital planning and investment
control review requirements.[Footnote 29] While we excluded BSM
projects from our assessment,[Footnote 30] our findings related to
project oversight are relevant to the BSM program. We noted in
particular that IRS has a strong oversight program, implementing all
of the seven associated key practices defined in our framework. For
example, consistent with these practices, IRS has developed written
policies and procedures for management oversight of its investments.
These include (1) a tiered escalation guide which outlines the process
for elevating a project to a higher level of control or governance for
review, mitigation, and resolution when resolution cannot be reached
at a project's respective level of control or governance; and (2)
written procedures and a template for conducting milestone exit
reviews to assess a project's readiness for moving to the next phase
of its life cycle or exit a milestone. In addition, the agency has
adequate resources for overseeing IT projects, including multiple
levels of governance boards. The agency also maintains an automated
system for tracking project action items assigned during governance
board meetings until mitigated. IRS also requires project management
plans that document cost, schedule, benefit, and risk expectations.
Consistent with OMB capital planning and investment control review
requirements, IRS also submits capital asset plans (known as exhibit
300s) for the BSM projects and also includes these projects in the
Treasury IT Investment Portfolio (generally referred to as the exhibit
53).
2. IRS's expenditure plan complies with IRS's EA:
In September 2010 IRS issued its Enterprise Transition Plan, a roadmap
for transitioning the IRS EA over the next 3 to 5 years. The plan,
which is updated annually, provides information on IRS's overall plan
to evolve IT and facilitate the investment decision-making process.
For example, the transition plan identifies key IT initiatives, such
as CADE 2, and identifies how they fit into operational goals deployed
over a 5-year time frame. Further, the BSM expenditure plan includes
activities necessary for ensuring that the BSM programs comply with
the IRS EA. Specifically, the plan identifies funding for continued
definition and implementation of the EA. For example, the plan
identifies funding needed for:
* performing architecture, engineering, and integration activities to
ensure that the IRS EA provides the information and guidance necessary
for modernization projects;
* supporting the performance of EA compliance certification
activities; and:
* finalizing and publishing updates to the EA based on change requests.
In addition, IRS has outlined general processes that projects are to
follow in order to ensure their compliance with the IRS EA.
Specifically, these processes discuss the activities and documentation
required to ensure compliance with the EA. For example, at milestone 3
in IRS's enterprise life cycle--which is the phase where a project's
preliminary design is developed--IRS requires projects to submit an EA
compliance checklist, which is used to help ensure that the project
design has been checked for compliance.
3. IRS's expenditure plan conforms to the agency's enterprise life
cycle methodology:
IRS has a documented enterprise life cycle methodology, which defines
a set of processes that are to be used throughout the program's life
cycle, including processes for managing system investments,
configuration, and risks. In August 2009, IRS revised its enterprise
life cycle methodology. The new guidance provided additional
information and clarification on topics such as project charters,
milestone readiness reviews, and obtaining concurrence for deviations
from the enterprise life cycle when tailoring the methodology for
specific projects. The BSM expenditure plan conforms to IRS's
enterprise life cycle. Specifically, the plan calls for the following:
* continuing to provide centralized guidance, administration,
mitigation, and closure of risks and issues throughout the life cycle
of each project;
* improving usability and tailoring standards for the enterprise life
cycle;
* building a central enterprise integration toolkit site for all
program management process tasks, assets, and tools on the enterprise
life cycle path; and:
* refining enterprise life cycle processes and procedures to address
gaps and weaknesses identified in the CMMI® level 2 appraisal of
application development for all BSM projects.[Footnote 31]
In addition, although reporting on funding for some program milestones
has been combined in previous years, the plan, consistent with our
recommendation, reports available performance information for
individual milestones according to their approved tailored enterprise
life cycle.
4. IRS's expenditure plan complies with the acquisition rules,
requirements, guidelines, and systems acquisition management practices
used by the federal government[Footnote 32]
The quality of software is governed largely by the quality of the
processes involved in developing or acquiring it and maintaining it.
Carnegie Mellon University's SEI[Footnote 33] calls for disciplined
software development and acquisition practices which are considered
industry best practices. SEI, recognized for its expertise in software
processes, has developed models and methods that define and determine
organizations' software process maturity, including the CMMI model
used to develop processes needed for software development and
acquisition and specific practices that agencies should follow to
mature these processes.
IRS initiated a CMMI implementation project to improve its software
acquisition and development practices by conducting a gap analysis--
which compared actual performance with potential performance--and
performing benchmarking activities, according to agency officials, in
June 2009. As a result of these activities, IRS developed process
improvement recommendations, including, among other things, updating
the existing project management plan template, expanding performance
monitoring, and establishing a communications management plan. In
November 2010, the MITS Application Development office was found to be
operating at the Managed Maturity Level 2 for both major and non-major
projects.[Footnote 34] As part of the assessment, reviewers looked at
IRS's software development and acquisition areas. Examples of
weaknesses identified include the need for targeted training for
Contracting Officers Technical Representatives to prepare them for
their responsibilities, as well as ensuring that all projects
establish and maintain a top-level work breakdown structure suitable
to scope and estimate their projects. IRS plans to address the
weaknesses as part of efforts to achieve a Level 3 designation.
In addition, the expenditure plan states that IRS will take the
following actions in fiscal year 2011:
* as part of the BSM management processes, the plan states that IRS
will collect earned value data on all major acquisitions as defined by
the Department of the Treasury, implement simplified earned value
reporting for all other projects, and integrate project schedules;
* develop process artifacts in process management, project management,
support management, and engineering management in order to incorporate
CMMI and other process improvements; and:
* refine the agency's enterprise life cycle processes and procedures
to address gaps and weaknesses identified in the CMMI level 2
assessment.
5. IRS's expenditure plan was approved by IRS, the Department of the
Treasury, and OMB:
The expenditure plan was reviewed and approved by IRS on August 18,
2010, the Department of Treasury on September 10, 2010, and OMB on
October 15, 2010. However, IRS revised the requested amounts for the
BSM programs after the enactment of the fiscal year 2011
appropriations. The revised request was reviewed and approved by IRS
on April 20, 2011; the Department of the Treasury on April 28, 2011;
and OMB on May 2, 2011.
6. IRS's expenditure plan was reviewed by GAO:
We completed our review of the plan in July 2011.
Results: Prior Recommendation Status:
IRS Has Implemented Four of Our Five Outstanding Recommendations from
Prior Expenditure Plan Reviews, but Still Needs to Provide a
Quantitative Measure of Progress in Meeting Project Scope Expectations"
IRS implemented our recommendations to revisit the BSM vision and
strategy and develop goals and plans, ensure explanations of cost and
schedule variances included underlying causes, report available
performance information for individual milestones, and define planning
activities for the second phase of the CADE 2 program. However, steps
remain to fully implement the recommendation to develop a quantitative
measure of progress in meeting project scope expectations.
Table 3: Status of IRS's Progress in Implementing Prior Expenditure
Plan Review Recommendations:
Prior GAO recommendation: Modernization vision and strategy--Fully
revisit the vision and strategy for the BSM program and develop a new
set of long-term goals, strategies, and plans that are consistent with
the budgetary outlook and IRS's management capabilities;
Implemented.
Prior GAO recommendation: Quantitative measures of progress in meeting
scope expectations--Ensure that future expenditure plans include a
quantitative measure of progress in meeting project scope expectations;
In progress.
Prior GAO recommendation: Underlying causes of variance--Ensure that
explanations of project cost and schedule variances provided in the
expenditure plan consistently include underlying causes;
Implemented.
Prior GAO recommendation: Reporting on combined milestones--For
individual milestones that have been combined, report available
performance information in the expenditure plan to provide Congress
with information on progress in delivering functionality;
Implemented.
Prior GAO recommendation: Time frames for CADE 2 transition state 2--
Define specific time frames for the second phase of CADE 2 planning
activities, including defining a core set of requirements to guide
progress;
Implemented.
Source: GAO analysis of IRS data.
[End of table]
Modernization Vision and Strategy:
In July 2005, we recommended that IRS fully revisit the vision and
strategy for the BSM program and develop a new set of long-term goals,
strategies, and plans consistent with the budgetary outlook and IRS's
management capabilities.[Footnote 35] We also noted that the vision
and strategy should include time frames for consolidating and retiring
legacy systems. IRS agreed with our recommendation and, in response,
took several actions. For example, IRS developed an initial cycle of
its Modernization Vision and Strategy in fiscal year 2006 and a
companion Enterprise Transition Plan to guide IT investment decisions
during fiscal years 2007 through 2011. The Enterprise Transition Plan,
which IRS has been updating on an annual basis, describes the overall
IRS IT modernization vision, the IT modernization strategies, and
existing and proposed investments that align with these strategies.
IRS also developed a high-level strategy and 5-year schedule for
retiring and consolidating systems, which it began implementing.
[Footnote 36] The most recent version of the Enterprise Transition
Plan also identifies IRS projects or systems that have been retired in
the last year, or that have been identified as potential retirements
in the next 1 to 5 years by current modernization and enhancement
projects.
IRS has also been building its internal management capabilities in
order to reduce reliance on vendors and has started shifting
responsibilities accordingly. For example, when BSM was initiated, IRS
contracted with Computer Sciences Corporation as the prime systems
integration support contractor for the program. According to IRS, the
agency itself now serves as the systems integrator for all BSM
systems, including Current CADE, MeF, and CADE 2.
Prompted by challenges facing the CADE program, IRS performed a
comprehensive study of its modernization program in 2008 which led to
a focus on CADE 2, IRS's revised strategy to address the management of
individual taxpayer accounts as well as several long-term goals to
enhance IRS's systems. Specifically, the strategy's initial phase is
to create a modernized taxpayer database and to move the processing of
individual taxpayer accounts from a weekly processing cycle to a daily
processing cycle by the January 2012 filing tax season. At the time of
our last expenditure plan review, we had just received several
planning documents for CADE 2 and had not yet assessed them to
determine if the program addressed our recommendation. We have since
reviewed these documents and performed a review of CADE 2.[Footnote
37] Based on our review of the program, and the efforts identified
above, we believe IRS has addressed our recommendation.
Quantitative Measure of Progress in Meeting Scope Expectations:
In February 2007, we recommended that IRS ensure that future
expenditure plans include a quantitative measure of progress in
delivering systems' planned functionality (scope).[Footnote 38] We
also recommended that, in developing this measure, IRS consider using
earned value management[Footnote 39] since this is a proven technique
required by OMB for measuring cost, schedule, and functional
performance (i.e., scope of work) against plans. While IRS agreed with
our recommendation to develop a quantitative measure of progress in
meeting scope expectations, it stated at the time that it did not
believe earned value management would provide this measure, given the
manner in which the technique was being used at the agency.[Footnote
40] Instead IRS proposed a two-step approach to address our
recommendation. As an initial step, IRS developed a qualitative
measure that indicated the difference between a project release's
planned and delivered capabilities and began using it in the fiscal
year 2008 expenditure plan. In March 2008 we reported that, as a
second step, IRS planned to leverage its requirements management tools
to assign quantitative values to the capabilities in order to develop
a quantitative measure of scope in the fiscal year 2009 plan. IRS did
not have the quantitative measure ready as planned but has continued
to work on its development.
According to the Chief Technology Officer, IRS now plans to include
this measure in the fiscal year 2012 expenditure plan. As we have
previously reported,[Footnote 41] fully implementing a quantitative
measure of progress in delivering systems' planned functionality will
help IRS in managing and controlling BSM. In addition, without this
measure, Congress may not have the information it needs to effectively
assess IRS's performance in implementing BSM.
Underlying Causes of Variance:
In May 2010, we recommended that IRS ensure that explanations of
project cost and schedule variances provided in the expenditure plan
consistently include underlying causes.[Footnote 42] We noted that
although IRS's guidance calls for an explanation of underlying causes,
this explanation was not included in the expenditure plan for
milestones that went over planned costs. In response, IRS stated that
by September 2010 it would revise its guidance and ensure that BSM
budget points of contact were notified of the updated guidance. In
October 2010, IRS included underlying causes of cost variances for the
five milestones in the fiscal year 2011 expenditure plan. For example,
for a milestone of the MeF program, IRS indicated there was a cost
variance of about $29 million. IRS stated that this was due to
implementing unplanned requirements including disaster recovery,
automated interface to support external users, and expanded hardware
needs. Reporting on the underlying causes of variances in the
expenditure plan provides Congress with insight into specific areas
needing improvement.
Reporting on Combined Milestones:
In May 2010, we recommended that for individual milestones that have
been combined, IRS should report available performance information in
the expenditure plan to inform Congress on progress in delivering
functionality.[Footnote 43] Further, we noted that due to the
combination of milestones and the manner in which they were reported
within the expenditure plan, we were unable to calculate the cost and
schedule variance for one of the BSM programs. In response, IRS stated
that by February 2011 it would ensure that the expenditure plan
incorporated available information for delivering functionality on all
individual milestones that had been combined. In the fiscal year 2011
expenditure plan, for milestones previously reported together, IRS
provided cost information related to each individual milestone.
Further, IRS noted that funding for the previously combined program
releases was initially tracked and reported as a single milestone
segment; however, it had been broken out into separate segments for
consistency with the project's approved enterprise life cycle
tailoring plan. Reporting milestone performance data aligned to the
project's enterprise life cycle tailoring plan will provide Congress
with better insight into the progress made in delivering functionality.
Time Frames for the Second Phase of CADE 2:
In May 2010, we recommended that IRS define time frames for the second
phase of CADE 2 planning activities, including defining a core set of
requirements to guide progress.[Footnote 44] Specifically, we noted
that while IRS had planned to set up teams to work on requirements,
design principles, framework for applications and infrastructure,
engineering analysis, and prototypes during fiscal year 2010, it had
not yet defined associated time frames for completing these key
planning activities. In response, IRS stated that by April 2011 a plan
would be developed for launching the second phase activities. In
December 2010, IRS identified a planning executive for the phase and
began the planning process. In March 2011, IRS completed an initial
plan for this phase describing capabilities and scope, the general
approach for planning and delivering the phase, and planned results
and initial time frames. The time frames include milestones for
completing foundation planning activities for fiscal year 2011 (for
example, clarifying the phase's scope and preparing the program
management office) and "ballpark" estimates spanning three fiscal
years for completing the phase.
Results: Observations:
Observations about IRS‘s BSM Program and Expenditure Plan:
Observation 1: Of 12 project milestones[Footnote 45] planned for
completion between January 2010 and May 2011, 11 were completed within
10 percent of schedule and cost estimates, under budget, or early, and
1 was completed on schedule but significantly over cost.
Our analysis of IRS's reported planned and actual milestone costs and
completion dates showed that 11 milestones were completed within 10
percent of cost and schedule estimates, under budget, or early, and 1,
while it was within 10 percent of schedule estimates, significantly
exceeded its cost estimate. Specifically, Release 6.1 of MeF, intended
to roll out the first phase of the 1040 processing capability,
completed a combined detailed design (milestone 4a), system
development (milestone 4b), and system deployment (milestone 5)
milestone more than 70 percent over estimated costs.
Figure 1 depicts the detailed cost and schedule variances of the
project milestones that were completed from January 2010 through May
2011.
Figure 1: Cost and Schedule Variances for Project Milestones Completed
from January 2010 through May 2011:
[Refer to PDF for image: vertical bar graph]
Plus or minus 10 percent is within acceptable range.
Milestone: 5;
Release: 2.1;
Program: AMS;
Cost variance: -63.7%.
Schedule variance: 0.
Milestone: 4b;
Release: 5.2;
Program: Current CADE;
Cost variance: -11.9%;
Schedule variance: -5.4%.
Milestone: 4b;
Release: 6.2;
Program: Current CADE;
Cost variance: 0;
Schedule variance: -6.0%.
Milestone: 3;
Release: TS1;
Program: CADE 2;
Cost variance: 0;
Schedule variance: -0.9%.
Milestone: 4a;
Release: TS1;
Program: CADE 2;
Cost variance: 0;
Schedule variance: -5.0%.
Milestone: 4a-5;
Release: 6.1;
Program: MeF;
Cost variance: 73.1%;
Schedule variance: 1.5%.
Milestone: 3-4a;
Release: 6.2;
Program: MeF;
Cost variance: 0;
Schedule variance: 0.
Milestone: 4b;
Release: 6.2;
Program: MeF;
Cost variance: 0;
Schedule variance: 0.
Milestone: 5;
Release: 6.2;
Program: MeF;
Cost variance: 0;
Schedule variance: 1.0%.
Milestone: 2;
Release: 7;
Program: MeF;
Cost variance: -42.4%;
Schedule variance: 0.
Milestone: 3;
Release: 7;
Program: MeF;
Cost variance: -23.6%;
Schedule variance: 0.
Milestone: 4a;
Release: 7;
Program: MeF;
Cost variance: 0;
Schedule variance: 0.
Source: GAO analysis of IRS data.
[End of figure]
The following provides details for the 11 milestones that were
completed within 10 percent of cost and schedule estimates, under
budget, or early:
Accounts Management Services (AMS):[Footnote 46]
* Release 2.1, milestone 5: this release, designed to provide
authorized employees an interface to view and update certain taxpayer
information, completed deployment on schedule at a cost of $398
thousand, or about 64 percent under budget. IRS indicated that the
variance was due to the required realignment of funds to support
Release 1.3 software and infrastructure design activities. IRS further
stated that this realignment was made possible by greater reliance on
less costly in-house government support and the deployment of Release
2.1 going more smoothly than originally anticipated.
Current CADE:
* Release 5.2, milestone 4b: this release was intended to implement
changes to the tax laws for filing season 2010 as well as deliver the
refund hold capability and the ability to process returns that include
credit election that were deferred from Release 4.2. The release
completed system development 9 days ahead of schedule at a cost of
about $19.6 million. This is $2.6 million (about 12 percent) less than
the amount budgeted for this milestone. IRS indicated that the costs
for this milestone were less than planned because the expected impacts
of changes to tax laws and the filing season requirements were reduced
in scope and complexity to a greater extent than anticipated.
* Release 6.2, milestone 4b: this release, intended to include tax law
and filing season changes, completed the development, test, and
integration phase at a cost of $22 million--as budgeted--and 10 days
earlier (6 percent) than planned.
CADE 2:
* Transition state 1: this phase of CADE 2 is intended to establish a
single database to house all individual taxpayer accounts, enhance
processing to include batch daily processing, provide more timely data
to the key customer service database, and provide data and tools to
allow business users to be more effective in compliance and customer
service.
- The logical design phase (milestone 3) was completed at a cost of
$14.2 million--as budgeted--and 2 days (0.9 percent) earlier than
planned.
- The physical design phase (milestone 4a) was completed at a cost of
$10 million--as budgeted--and 9 days (6 percent) earlier than planned.
MeF:
* Release 6.2: this release, intended to enhance the disaster recovery
capabilities provided in Release 6.1 and also to implement code
optimization and performance enhancement to support the volume of
returns anticipated for 2012, completed three milestones during the
reporting period.
- The design phase (milestone 3-4a) was completed on schedule at a
cost of $8.9 million--as budgeted.
- The release completed system development, integration, and testing
(milestone 4b) on December 29, 2010, as planned and at a cost of $7.8
million--as budgeted.
- System deployment (milestone 5) was completed one day later (1
percent) than planned, and at a cost of $5.2 million--as budgeted.
* Release 7: this release, when deployed in 2012, is intended to roll
out over 125 remaining 1040 family schedules and forms, including
forms 1040A and 1040EZ, which will expand MeF's reach to 100 percent
of the e-file population, or approximately 98.3 million filers. IRS
completed both conceptual and preliminary design phases on schedule
and under budget, while detailed design met both cost and schedule
estimates.
- Conceptual design phase (milestone 2), budgeted to cost about $5.3
million, was completed on schedule at a cost of approximately $3
million (or about 42 percent under budget).
- Preliminary design (milestone 3), budgeted at about $15.9 million,
was completed on schedule at a cost of approximately $12 million (or
about 24 percent under budget).
- Detailed design (milestone 4a), was completed on schedule at a cost
of about $11.8 million--as budgeted.
The following provides information on the MeF milestone that was
completed on schedule, but exceeded planned costs by more than 70
percent. As we noted last year,[Footnote 47] the deployment of Release
6.1 of MeF represented a change in strategy for the program.
Specifically, in response to schedule delays due to added complexities
and requirements, IRS developed a new deployment strategy whereby the
deployment of the 1040 series of forms was expected to be completed up
to 2 years later than planned under the initial strategy and at a cost
of nearly $50 million more:
* Release 6.1: this release, which represented the first phase of the
IRS rollout of the MeF 1040 capability, completed the deployment phase
(milestone 5) 7 days later than scheduled at a cost of about $68.9
million--approximately 73 percent over the budget of approximately
$39.8 million. According to IRS, several factors contributed to the
overrun, including the need for additional funds to implement
unplanned requirements such as the first phase of disaster recovery,
Transactional National Account Profile, automated interface to support
external users, increased availability/code optimization, and expanded
hardware needs.
Observation 2: A large percentage of requested BSM funding is
categorized as level-of-effort work.
IRS is requesting about $174 million, or 49 percent of the total BSM
request, for level of effort (LOE) work.[Footnote 48] According to the
GAO Cost Guide, LOE consists of tasks of a general or supportive
nature, such as program management, which do not result in clear work
products.[Footnote 49] Therefore schedule variances are not reported
for such work. If more than 15 percent of a program's budget is
classified as level of effort, best practices call for this amount to
be scrutinized.
One factor contributing to the large percentage of funding for LOE
work requested in the BSM expenditure plan is that IRS's definition of
LOE differs from the commonly accepted definition in that it includes
activities that result in clear work products, such as a risk
management plan or monthly reports. For example, IRS is characterizing
the work associated with developing the CADE 2 infrastructure as LOE
even though this work is expected to include the development of
production and disaster recovery environments, which should include
discrete work.[Footnote 50] When we discussed this issue with IRS, we
found that the agency is managing some of the work assigned as LOE in
the expenditure plan as discrete work and accordingly tracking
schedule performance information for it. By categorizing discrete work
as LOE in the expenditure plan, IRS is (1) presenting an inaccurate
picture of the work it is performing and the manner in which it is
being managed, and (2) providing Congress with less insight into its
performance in implementing the BSM program.
Observation 3: CADE 2 risk management and preliminary cost estimating
processes are generally consistent with best practices; however, IRS
will be challenged in completing one of two key projects by January
2012 as planned. In addition, IRS has developed a baseline cost
estimate for the first phase of the CADE 2 program.
In March 2011, we reported that while IRS processes for managing risk
and estimating costs for the CADE 2 program were generally consistent
with best practices, the cost estimate could be improved and the
expected benefits had not yet been fully defined.[Footnote 51] As a
result, we recommended that IRS identify benefits for the second
phase, set related targets, and identify how systems and business
processes might be affected. We also recommended improving the
credibility of the cost estimate by (1) including inflation when
calculating costs, (2) including the costs that were explicitly
excluded or providing a rationale for excluding them, and (3)
including business costs associated with moving to daily processing or
document that these costs were excluded and provide a rationale for
excluding them. IRS agreed with our recommendations and provided
corrective actions and implementation dates for the recommendations.
We also reported that while IRS is working to ensure CADE 2 is
successfully managed, the schedule for delivering the initial phase
was ambitious. IRS officials acknowledged this and took actions to
increase their chances of meeting it, including moving certain
activities up, performing others concurrently, and adding checkpoints
to monitor the program's status. We noted while these actions may
increase the likelihood of meeting the schedule, some of them, such as
performing activities concurrently, could potentially introduce more
risk to CADE 2's successful development and implementation.
Further, in March 2011, IRS made the first of three critical decisions
planned for the first phase of CADE 2. According to IRS officials, the
first decision was whether or not to move from the current processing
environment to a "new cycle" and accelerated Individual Master File
for the fiscal year 2012 filing season.[Footnote 52] According to IRS,
the two remaining decision points--to be made in September 2011--will
be whether or not to move forward with (1) daily processing and (2)
database implementation.
To assist with the decision as to whether the program was ready to
proceed with the new cycle and accelerated Individual Master File, IRS
commissioned a readiness assessment by an independent group. In
February 2011, the reviewers reported that based on the likelihood of
success versus consequences of potential failure, IRS should proceed
with the new cycle and accelerated Individual Master File, largely due
to the available contingencies should the program falter. The
reviewers also assessed the other two decision points in the context
of how they related to the first. The review found that there was a
strong case for moving ahead with daily processing. Specifically, it
stated that the program was on track and should continue as planned.
However, the reviewers stated there was a lower likelihood of success
for the database implementation component due to compressed delivery
time frames and the steep learning curve required of new staff and
recommended actions to ensure success going forward. According to the
Deputy Chief Information Officer for Strategy and Modernization and
the Program Director for CADE 2, IRS is still on track for delivering
the database by January 2012 as planned. However, consistent with a
recommendation from the independent assessment, it is considering
phasing its implementation to reduce the impact on filing season
operations.
In addition, at the completion of preliminary design, IRS updated its
cost estimate for the first phase of the CADE 2 program. In this
updated estimate, IRS determined that the cost to complete this phase
was approximately $241 million.[Footnote 53] In contrast to the July
2009 preliminary estimate which we assessed as part of our CADE 2
review, the estimate is based on updated information resulting from
design activities. According to the Deputy Chief Information Officer
for Strategy and Modernization, this estimate will serve as the
baseline for the first phase of CADE 2.
Observation 4: Information security weaknesses continue to affect
IRS's modernization environment.
IRS continues to have information security weaknesses that affect its
modernization environment. In March 2011,[Footnote 54] we reported
that 65 out of the 88 previously reported information security
weaknesses that remained unresolved and new weaknesses identified were
due, in part, to the fact that IRS had not yet fully implemented key
components of its comprehensive information security program. Although
not all these weaknesses pertain to BSM specifically, IRS's
information security program encompasses all agency systems, and
therefore the program weaknesses affect the modernization environment.
For example, we reported that the agency's monitoring process for
verifying whether an action had corrected or mitigated an identified
weakness was not working as intended, and that IRS continued to inform
us that it had corrected reported weaknesses which we determined were
not yet fully resolved. We recommended that IRS take eight actions to
fully implement key components of its comprehensive information
security program. In June 2011, IRS provided us with the status of
actions taken and activities planned to address these recommendations.
For example, IRS stated it plans to enhance the monitoring and testing
of corrective actions after they have been implemented, and continue
monitoring them on an ongoing basis. While these completed and planned
actions are positive steps, we have not yet determined their
effectiveness in addressing our recommendations.
In September 2010,[Footnote 55] TIGTA reported that while IRS had
taken steps to address 16 CADE system security vulnerabilities it had
identified in a previous audit, only 10 had been fully resolved, and
the remaining 6 could not be resolved until actions were completed to
ensure controls are effectively in place or have been approved as
deviations to IRS policy. Further, TIGTA found that IRS prematurely
reported resolution of 6 vulnerabilities in its Plan of Action and
Milestones list before effective corrective actions were taken. Three
of these 6 vulnerabilities were not fully resolved as of the date of
TIGTA's review. IRS agreed with TIGTA's recommendations and stated
that the cybersecurity organization had made improvements to its
process to ensure that system owners comply with IRS policy to enter
and track all system vulnerabilities in IRS's tracking and reporting
control system. In addition to the steps that IRS is taking to address
security recommendations from GAO and TIGTA, IRS has stated that, in
transition state 2, CADE 2 will fully address the security weaknesses
that are currently confronting CADE. While actions to address the
identified report findings will help to improve its security posture,
IRS's modernization environment will continue to be at risk until the
agency fully implements its security program.
[End of section]
Conclusions:
IRS's fiscal year 2011 expenditure plan satisfies all six statutory
conditions. Further, IRS has made progress in addressing most of the
outstanding recommendations from our prior expenditure plan reviews.
However, IRS is still working to include a quantitative measure of
progress in meeting scope expectations in the plan, a recommendation
we made over 4 years ago. Until IRS fully implements this
recommendation, Congress may not have the information it needs to
effectively assess the implementation of the BSM effort.
During fiscal year 2010, IRS successfully completed 11 out of 12
project milestones within 10 percent of cost and schedule, under
budget, or early. However, one milestone was completed more than 70
percent over cost due to unplanned requirements. Further, IRS has
categorized a large amount--about 50 percent--of the funding requested
in the expenditure plan as level of effort, including for work that
does not appear to be appropriately characterized as such. By not
ensuring that all funding is appropriately categorized, IRS is
presenting an inaccurate picture of the work it is performing and the
manner in which it is being managed and providing Congress with less
insight into its performance in implementing the BSM program.
IRS has also taken key steps to ensure that its CADE 2 program is
being successfully managed. However, the program's schedule is
ambitious, and this has been corroborated by a recent independent
assessment. IRS officials have stated they are still on track for
completing planned work on schedule but are taking steps to reduce
risks.
Finally, weaknesses in information security continue to affect IRS's
modernization environment, largely because the agency has not fully
implemented key components of a comprehensive information security
program. Both we and the Treasury Inspector General for Tax
Administration have reported on these weaknesses, and full
implementation of outstanding recommendations will be needed for IRS
to ensure the security of its systems and the information they contain.
[End of section]
Recommendations for Executive Action:
To more accurately represent the work IRS is performing and provide
Congress greater insight into the agency's performance in implementing
BSM, we recommend that the Commissioner of Internal Revenue direct the
Chief Technology Officer take the following three actions,
(1) modify IRS's definition of level of effort work to be consistent
with the commonly accepted definition;
(2) reclassify activities involving discrete work and accordingly
report on associated performance data in future expenditure plans; and:
(3) consistent with best practices, scrutinize any remaining LOE work
exceeding 15 percent of the program's budget to look for opportunities
to reduce the amount.
[End of section]
Agency Comments and Our Evaluation:
In e-mail comments on a draft of these briefing slides, IRS officials,
including the Deputy Chief Information Officer for Strategy and
Modernization, agreed with our recommendation to accurately represent
the work IRS is performing and provide Congress with ongoing insight
into the agency's performance in implementing BSM. However, the agency
stated it did not agree with the actions we specified it take to
implement our recommendation: IRS did not specifically address reasons
for this disagreement. Nevertheless, IRS stated it would like to work
with us on how to best implement our recommendation to ensure an
adequate level of insight into implementation of the BSM program. IRS
also provided technical comments which we have incorporated as
appropriate.
Appendix I: Description of BSM Projects and Program-Level Initiatives:
Tax administration projects:
Proposed modernization initiative: Modernized e-File (MeF);
Description: Is to provide a single standard for filing electronic tax
returns. Initial releases will address large corporations, small
business, and tax-exempt organizations. Its ultimate goal is the
conversion of IRS's 1040 e-file program.
Proposed modernization initiative: Current Customer Account Data
Engine (CADE);
Description: Is to build the modernized database foundation to replace
the existing Individual Master File processing system that contains
the repository of individual taxpayer information.
Proposed modernization initiative: Customer Account Data Engine 2
(CADE 2);
Description: Is intended to leverage CADE and the Individual Master
File, which contain the repository of individual taxpayer information,
to provide timely access to authoritative individual taxpayer account
information and enhance IRS's ability to address technology, security,
financial material weaknesses, and long-term architectural planning
and viability.
Core infrastructure projects:
Proposed modernization initiative: Development, Integration, and
Testing Environments;
Description: Is to provide oversight for laboratory environments that
support evaluation, development, and testing of components from
multiple projects: (1) Virtual Development Environment intended to
provide a software development environment and a standardized set of
tools, and (2) Enterprise Integration and Test Environment intended to
provide an integration and testing environment for all projects.
Proposed modernization initiative: Infrastructure Shared Services;
Description: Is to deliver, in incremental releases over multiple
years, a fully integrated, shared IT infrastructure to include
hardware, software, shared applications, data, telecommunications,
security, and an enterprise approach to systems and operations
management.
Architecture, integration, and management:
Proposed modernization initiative: Architecture and integration;
Description: Is to ensure that systems solutions meet IRS business
needs and that the development projects are effectively integrated
into the business environment.
Proposed modernization initiative: Program governance and control;
Description: Is to facilitate the alignment of portfolio governance
and control functions, ensuring the consistent application of
appropriate strategic planning, investment management, and deployment
practices across all IT projects. Previous expenditure plans referred
to this as Business Integration.
Proposed modernization initiative: Requirements management;
Description: Is to ensure that the BSM program is aligned with the
business units' vision to deliver the desired business results by
providing a comprehensive requirement methodology, standards, and
guidance.
Proposed modernization initiative: Management processes;
Description: Is to provide sustaining support for program-level
management processes, including quality assurance, risk management,
program control and process management, and enterprise life cycle
maintenance and enhancements.
Proposed modernization initiative: Federally funded research and
development center;
Description: Is to provide program management and systems engineering
support.
Proposed modernization initiative: Program management;
Description: Is to ensure programs achieve their objectives by
providing the management information and IT infrastructure that
support risk management, project cost and schedule estimating,
financial management, and procurement management for the prime
contracts and associated task orders.
Source: GAO analysis of IRS data.
[End of table]
[End of section]
Appendix II: Additional Detail on IRS's Fiscal Year 2010 BSM
Expenditure Plan:
Tax administration projects:
Proposed modernization initiative: MeF;
Release[A]: 7;
Milestone/type of activity[B]: 4b;
Amount: $22,100,000.
Proposed modernization initiative: MeF;
Release[A]: 7;
Milestone/type of activity[B]: 5;
Amount: $7,000,000.
Proposed modernization initiative: MeF;
Release[A]: 8;
Milestone/type of activity[B]: 2-3;
Amount: $10,000,000.
Subtotal--MeF project;
Amount: $39,100,000.
Proposed modernization initiative: Current CADE;
Release[A]: 7.2;
Milestone/type of activity[B]: 4b;
Amount: $13,000,000.
Proposed modernization initiative: Current CADE;
Release[A]: Operations and maintenance;
Milestone/type of activity[B]: Level of effort;
Amount: $15,000,000.
Subtotal--CADE project;
Amount: $28,000,000.
Proposed modernization initiative: CADE 2;
Release[A]: Infrastructure;
Milestone/type of activity[B]: Level of effort;
Amount: $77,000,000.
Proposed modernization initiative: CADE 2;
Release[A]: Program management office;
Milestone/type of activity[B]: Level of effort;
Amount: $8,700,000.
Proposed modernization initiative: CADE 2;
Release[A]: Engineering and analysis;
Milestone/type of activity[B]: Level of effort;
Amount: $4,700,000.
Proposed modernization initiative: CADE 2;
Release[A]: Transition state 1;
Milestone/type of activity[B]: 4b;
Amount: $36,700,000.
Proposed modernization initiative: CADE 2;
Release[A]: Transition state 2;
Milestone/type of activity[B]: 3;
Amount: $0.
Proposed modernization initiative: CADE 2;
Release[A]: Up/downstream;
Milestone/type of activity[B]: Level of effort;
Amount: $9,000,000.
Subtotal--CADE 2 project;
Amount: $136,100,000.
Subtotal--tax administration projects;
Amount: $203,200,000.
Core infrastructure projects:
Proposed modernization initiative: Development, integration, and
testing environment;
[Empty];
Milestone/type of activity[B]: Infrastructure;
Amount: $14,500,000.
Proposed modernization initiative: Infrastructure shared services;
[Empty];
Milestone/type of activity[B]: Infrastructure;
Amount: $17,500,000.
Subtotal--core infrastructure projects;
Amount: $32,000,000.
Architecture, integration, and management:
Proposed modernization initiative: Architecture and integration;
Milestone/type of activity[B]: Level of effort;
Amount: $11,200,000.
Proposed modernization initiative: Program governance and control;
Milestone/type of activity[B]: Level of effort;
Amount: $1,650.
Proposed modernization initiative: Requirements management;
Milestone/type of activity[B]: Level of effort;
Amount: $2,670,000.
Proposed modernization initiative: Management processes;
Milestone/type of activity[B]: Level of effort;
Amount: $3,250,000.
Proposed modernization initiative: Federally funded research and
development center;
Milestone/type of activity[B]: Level of effort;
Amount: $6,050,000.
Proposed modernization initiative: Program management;
Milestone/type of activity[B]: Level of effort;
Amount: $2,735,000.
Subtotal--architecture, integration, and management;
Amount: $27,555,000.
Management reserve;
Amount: $8,925,000.
BSM capital total;
Amount: $271,680,000.
BSM labor total;
Amount: $80,689,000.
Maintaining current levels;
Amount: $0.
Total fiscal year 2011 BSM program;
Amount: $352,369,000.
Source: GAO analysis of IRS data.
[A] Releases are software versions that provide a subset of the total
planned project functionality.
[B] Milestones correspond to phases within IRS's enterprise life cycle
(1-project initiation, 2-conceptual design or business architecture, 3-
system logical design/system architecture, 4a-completion of system
physical design and 4b-development, test and integration, 5-system
deployment and transition to support and operations). According to
IRS, "level of effort" activities generally support the overall
management of the program, such as contracting or risk management.
They do not follow the enterprise life cycle methodology nor do they
directly create end user benefits. They do require the contractor to
provide a specified level of effort, over a stated period, on work
stated in general terms. Infrastructure projects follow many aspects
of the enterprise life cycle but differ from development projects in
that they (1) do not have a separate business case, though they have
definable costs and general broad benefits; (2) operate based on the
support needs of the project, rather than end-user requirements; (3)
cannot make changes that are independent from the projects they
support, that is, the infrastructure projects must synchronize changes
with the application projects and assess the impact; and (4) project
teams have no direct control over external changes (e.g., a new
requirement for an application project) that will drive cost
fluctuations.
[End of table]
[End of section]
Appendix III: IRS Reported Cost and Schedule Variance for Project
Milestones:
This table shows project segments completed from January 2010 through
May 2011.
Project segment: AMS; Release 2.1; Milestone 5;
Estimated completion date and funding: 02/08/10; $1,097,000;
Milestone exit and cost: 02/08/10; $398,000;
Change: 0 days; (0%); -$699,000; (-63.7%);
IRS explanation of change: IRS reported spending was less on this
milestone due to the required realignment of funds to support Release
1.3 software and infrastructure design activities, the greater
reliance on less costly in-house government support, and the fact that
the deployment of Release 2.1 went more smoothly than originally
anticipated.
Project segment: CADE; Release 5.2; Milestone 4b;
Estimated completion date and funding: 01/31/10; $22,200,000;
Milestone exit and cost: 01/15/10; $19,565,000;
Change: -10 days; (-6.0%); -$2,635,000; (-11.9%);
IRS explanation of change: IRS indicates that the costs for this
milestone were less than planned because the expected impacts of
legislative and filing season change requirements were reduced in
scope and complexity.
Project segment: CADE; Release 6.2; Milestone 4b;
Estimated completion date and funding: 01/31/11; $22,000,000;
Milestone exit and cost: 01/14/11; $22,000,000;
Change: -11 days; (-6.6%); $0; (0%);
IRS explanation of change: [Empty].
Project segment: CADE 2; Transition state 1; Milestone 3;
Estimated completion date and funding: 12/15/10; $14,200,000;
Milestone exit and cost: 12/13/10; $14,200,000;
Change: -2 days; (-0.9%); $0; (0%);
IRS explanation of change: [Empty].
Project segment: CADE 2; Transition state 1; Milestone 4a;
Estimated completion date and funding: 04/30/11; $10,000,000;
Milestone exit and cost: 04/18/11; $10,000,000;
Change: -9 days; (-6.0%); $0; (0%);
IRS explanation of change: [Empty].
Project segment: MeF; Release 6.1; Milestone 4a-5;
Estimated completion date and funding: 05/14/10; $39,802,000;
Milestone exit and cost: 05/25/10; $68,902,000;
Change: 7 days; (1.5%); $29,100,000; (73.1%);
IRS explanation of change: IRS pointed to several factors contributing
to the overrun, including the need for additional funds to implement
unplanned requirements including the first phase of disaster recovery,
Transactional National Account Profile, automated interface to support
external users, increased availability/code optimization, and expanded
hardware needs.
Project segment: MeF; Release 6.2; Milestone 3-4a;
Estimated completion date and funding: 09/21/10; $8,900,000;
Milestone exit and cost: 09/21/10; $8,900,000;
Change: 0 days; (0%); $0; (0%);
IRS explanation of change: [Empty].
Project segment: MeF; Release 6.2; Milestone 4b;
Estimated completion date and funding: 12/29/10; $7,800,000;
Milestone exit and cost: 12/29/10; $7,800,000;
Change: 0 days; (0%); $0; (0%);
IRS explanation of change: [Empty].
Project segment: MeF; Release 6.2; Milestone 5;
Estimated completion date and funding: 05/17/11; $5,200,000;
Milestone exit and cost: 05/18/11; $5,200,000;
Change: 1 day; (1.0%); $0; (0%);
IRS explanation of change: [Empty].
Project segment: MeF; Release 7; Milestone 2;
Estimated completion date and funding: 07/27/10; $5,295,000;
Milestone exit and cost: 07/27/10; $3,049,000;
Change: 0 days; (0%); -$2,246,000; (-42.4%);
IRS explanation of change: IRS stated that the cost variance was the
result of lower-than-expected hardware and software costs.
Project segment: MeF; Release 7; Milestone 3;
Estimated completion date and funding: 12/21/10; $15,875,000;
Milestone exit and cost: 12/21/10; $12,121,000;
Change: 0 days; (0%); -$3,754,000; (-23.6%);
IRS explanation of change: IRS stated that the cost variance was the
result of lower-than-expected hardware and software costs.
Project segment: MeF; Release 7; Milestone 4a;
Estimated completion date and funding: 04/26/11; $11,830,000;
Milestone exit and cost: 04/26/11; $11,830,000;
Change: 0 days; (0%); $0; (0%);
IRS explanation of change: [Empty].
Source: GAO analysis of IRS data.
Note: Variances for projects through milestone 3 are based on rough
order of magnitude estimates. Post-milestone-3 variances are based on
more specific estimates.
[End of table]
[End of section]
Web site: [hyperlink, http://www.gao.gov/].
Chuck Young, Managing Director, Public Affairs, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
This is a work of the U.S. government and is not subject to copyright
protection in the United States. The published product may be
reproduced and distributed in its entirety without further permission
from GAO. However, because this work may contain copyrighted images or
other material, permission from the copyright holder may be necessary
if you wish to reproduce this material separately.
[End of briefing slides]
Appendix II: Comments from the Internal Revenue Service:
Department Of The Treasury:
Internal Revenue Service:
Commissioner:
Washington, DC 20224:
September 22, 2011:
Mr. David Powner:
Director, Information Technology Management Issues:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, D.C. 20548:
Dear Mr. Powner:
Thank you for the opportunity to comment on the Government
Accountability Office (GAO) draft report titled "Business Systems
Modernization: Internal Revenue Service's Fiscal Year 2011 Expenditure
Plan,“ (GAO-12-26). The Business Systems Modernization (BSM) Program
emphasizes development and delivery of a number of modernized tax
systems and is intended to replace the Agency's aging tax processing
systems. This remains one of the IRS's top priorities.
As GAO recognizes, performing a major technology enhancement to core
taxpayer account systems that process approximately $2.5 trillion in
annual revenues is a highly complex effort. We must continue to invest
in BSM to maintain our country's tax system, which contributes to the
overall economic infrastructure.
I appreciate that your report recognizes that our expenditure plan
satisfies each of the six applicable statutory conditions set out by
Congress, including meeting the Office of Management and Budget's
capital planning and investment control review requirements, and
complying with federal systems acquisition requirements and management
practices. We are also proud of the fact that the IRS has achieved the
Software Engineering Institute's Level 2 for Capability Maturity Model
Integration, and is well on the way to achieving Level 3.
We continue to value GAO's feedback, and have implemented four of the
five outstanding recommendations from prior expenditure plan reviews.
We plan to implement the fifth recommendation, the inclusion of
quantitative measures of progress in meeting project scope
expectations, as a metric in our FY 2012 Expenditure Plan.
I also understand that IRS staff have expressed disagreement over
certain aspects of your feedback on cost estimation but continue to
engage in constructive dialog to ensure that the level of effort the
IRS puts into specific deliverables is clearly expressed in our
expenditure plan.
Finally, in addressing your concern in the area of information
security weaknesses, we consider the security of our systems and
information assets to be of paramount importance to the execution of
our mission. Further through ongoing efforts to preserve and expand
the security of our systems and information, we have made substantial
progress in the resolution of identified security concerns over the
past year. In light of which, as you note in your report, we have
requested your review of the actions we have completed to address your
eight recommendations to fully implement our comprehensive information
security program.
We appreciate your continued support and guidance. If you have any
questions, please contact me or a member of your staff may contact
Terence V. Milholland, Chief Technology Officer, at (202) 622-6800.
Sincerely,
Signed by:
Douglas H. Shulman:
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
David A. Powner, (202) 512-9286 or pownerd@gao.gov:
Staff Acknowledgments:
In addition to the individual named above, Sabine R. Paul, Assistant
Director; Sairah R. Ijaz; Lee A. McCracken; and Paul B. Middleton made
key contributions to this report.
[End of section]
Footnotes:
[1] The expenditure plan was prepared in October 2010; however, IRS
revised its request for the BSM program following the enactment of the
fiscal year 2011 funding in April.
[2] BSM funds (except labor costs) are unavailable until IRS submits a
modernization expenditure plan to the congressional appropriations
committees. See the Consolidated Appropriations Act, 2010, Pub. L. No.
111-117, div. C, title I, 123 Stat. 3034, 3164 (Dec. 16, 2009), under
the authority and conditions of which the Full-Year Continuing
Appropriations Act, 2011, Pub. L. No. 112-10, Division B, title I,
Sec. 1101(a)(6), 125 Stat. 38, 102-103 (April 15, 2011), appropriates
funds for the IRS for fiscal year 2011.
[3] This plan must (1) meet the Office of Management and Budget's
(OMB) capital planning and investment control review requirements; (2)
comply with IRS's enterprise architecture; (3) conform with IRS's
enterprise life cycle methodology; (4) comply with federal acquisition
rules, requirements, guidelines, and systems acquisition management
practices; (5) be approved by IRS, the Department of the Treasury, and
OMB; and (6) be reviewed by GAO. These conditions for BSM funding
availability have been in effect for several years, including the
immediately preceding fiscal year. At the time of this report, the
latest appropriation was provided under Pub. L. No. 112-10, Division
B, title I, Sec. 1101(a)(6), 125 Stat. 38, 102-103 (April 15, 2011);
see Pub. L. No. 111-117, div. C, title I, 123 Stat. 3034, 3164, (Dec.
16, 2009).
[4] For this condition, we did not determine whether the expenditure
plan comports with the Federal Acquisition Regulation or other federal
requirements beyond those encompassed by the Software Engineering
Institute's Capability Maturity Model Integration (CMMI).
[5] IRS's guidance defines milestones as management decision points
reviewing updated cost, progress, and risk information to make
continuation decisions. The milestones are: 1-project initiation
(vision and strategy), 2-conceptual design or business architecture, 3-
system logical design/system architecture, 4-development, test, and
integration, 5-system deployment and transition to support and
operations. The development, test, and integration phase is sometimes
split into two phases, each corresponding to a different milestone: 4a-
completion of system physical design and 4b-development, test, and
integration.
[6] GAO, Taxpayer Account Strategy: IRS Should Finish Defining
Benefits and Improve Cost Estimates, [hyperlink,
http://www.gao.gov/products/GAO-11-168] (Washington, D.C.: Mar. 24,
2011).
[7] According to IRS, this amount represents an increase of $132
million from the preliminary estimate issued in July 2009. The
preliminary estimate, however, was a rough-order-of-magnitude estimate
that was not informed by the detailed design information for the
Database Implementation and Daily Processing projects that has been
developed since.
[8] GAO, Information Security: IRS Needs to Enhance Internal Control
over Financial Reporting and Taxpayer Data, [hyperlink,
http://www.gao.gov/products/GAO-11-308] (Washington, D.C.: Mar. 15,
2011).
[9] Among other things, the approach to develop the system was more
complex and taking longer than initially anticipated.
[10] Consolidated Appropriations Act, 2010, Pub. L. No.111-117, div.
C, title I, 123 Stat. 3034, 3164 (Dec. 16, 2009), under the authority
and conditions of which the Full-Year Continuing Appropriations Act,
2011, Pub. L. No. 112-10, Division B, title I, Sec. 1101(a)(6), 125
Stat. 38, 102-103 (April 15, 2011), appropriates funds for the IRS for
fiscal year 2011. Therefore, the statutory conditions that applied in
fiscal year 2010 to IRS's expenditure plan continue to apply during
fiscal year 2011.
[11] These funds support IRS's efforts to modernize its business
systems. Other system efforts, including the maintenance of both BSM
and non-BSM applications, are typically supported by funding that is
not subject to the BSM program statutory conditions.
[12] An EA is an institutional blueprint that defines how an
enterprise operates today, in both business and technology terms, and
how it intends to operate in the future. An EA also includes a road
map for transitioning between these environments.
[13] IRS refers to its life cycle management program as the enterprise
life cycle.
[14] The expenditure plan was prepared in October 2010; however, IRS
revised its request for the BSM program following enactment of the
fiscal year 2011 funding.
[15] IRS's guidance defines milestones as management decision points
reviewing updated cost, progress, and risk information to make
continuation decisions. Milestone 1-project initiation (vision and
strategy), 2-conceptual design or business architecture, 3-system
logical design/system architecture, 4-development, test, and
integration, 5-system deployment and transition to support and
operations. The development, test, and integration phase is sometimes
split into two phases, each corresponding to a different milestone: 4a-
completion of system physical design and 4b-development, test, and
integration.
[16] GAO, Taxpayer Account Strategy: IRS Should Finish Defining
Benefits and Improve Cost Estimates, [hyperlink,
http://www.gao.gov/products/GAO-11-168] (Washington, D.C.: Mar. 24,
2011).
[17] GAO, Information Security: IRS Needs to Enhance Internal Control
over Financial Reporting and Taxpayer Data, [hyperlink,
http://www.gao.gov/products/GAO-11-308] (Washington, D.C.: Mar. 15,
2011).
[18] GAO, Business Systems Modernization: IRS Has Made Significant
Progress in Improving Its Management Controls, but Risks Remain,
[hyperlink, http://www.gao.gov/products/GAO-03-768] (Washington, D.C.:
June 27, 2003).
[19] GAO, Business Systems Modernization: IRS Needs to Complete Recent
Efforts to Develop Polices and Procedures to Guide Requirements
Development and Management, [hyperlink,
http://www.gao.gov/products/GAO-06-310] (Washington, D.C.: Mar. 20,
2006).
[20] GAO, Business Systems Modernization: IRS's Fiscal Year 2004
Expenditure Plan, [hyperlink, http://www.gao.gov/products/GAO-05-46]
(Washington, D.C.: Nov. 17, 2004).
[21] GAO, Business Systems Modernization: Internal Revenue Service's
Fiscal Year 2007 Expenditure Plan, [hyperlink,
http://www.gao.gov/products/GAO-07-247] (Washington, D.C.: Feb. 15,
2007).
[22] GAO, Business Systems Modernization: Internal Revenue Service's
Fiscal Year 2009 Expenditure Plan, [hyperlink,
http://www.gao.gov/products/GAO-09-281] (Washington, D.C.: Mar. 11,
2009).
[23] IRS's Chief Technology Officer, who was appointed in November
2008, reports to the Deputy Commissioner for Operations Support.
[24 Efforts to maintain and enhance non-BSM systems are funded from a
different appropriation not subject to the BSM expenditure plan
appropriations restriction.
[25] Current CADE refers to the prime contract development efforts
that began in 2002 to replace the legacy Individual Master File and to
date have resulted in a series of five releases and sub-releases. Up
until the time IRS starting defining CADE 2, it was referred to as
CADE.
[26] IRS officials have stated that the agency does not yet have any
plans for modernizing the Business Master File, which stores tax data
and related information on business taxpayers. However, IRS officials
have told us that they expect to be able to replicate the CADE 2
approach to modernize the Business Master File.
[27] Full-Year Continuing Appropriations Act, 2011, Pub. L. No. 112-
10, Division B, Title I, Sec. 1101(a)(6), April 15, 2011, 125 Stat.
38, 102-103; and Consolidated Appropriations Act, 2010, Pub. L. No.
111-117, Division C, Title 1, Dec. 16, 2009, 123 Stat. 3034, 3164.
[28] GAO, Information Technology Investment Management: A Framework
for Assessing and Improving Process Maturity, [hyperlink,
http://www.gao.gov/products/GAO-04-394G] (Washington, D.C.: March
2004).
[29] GAO, Investment Management: IRS Has a Strong Oversight Process
but Needs to Improve How It Continues Funding Ongoing Investments,
[hyperlink, http://www.gao.gov/products/GAO-11-587] (Washington, D.C.:
to be released on July 20, 2011).
[30] BSM investments go through a different investment management
process than other investments at IRS. Because we have been reviewing
these investments for more than a decade and reporting on them on a
now-annual basis, we did not include them in the scope of our
investment management review.
[31] CMMI is registered in the U.S. Patent and Trademark Office by
Carnegie Mellon University. The CMMI is the Software Engineering
Institute's (SEI) model that describes how to develop the processes
needed for software development and specific practices that
organizations should follow.
[32] For this condition, we did not determine whether the expenditure
plan comports with the Federal Acquisition Regulation or other federal
requirements beyond those encompassed by the SEI's Capability Maturity
Model. Key process areas addressed in the model include acquisition
planning, solicitation, requirements development and management,
project management, contract tracking and oversight, evaluation, and
transition to support.
[33] SEI is a federally funded research and development center
operated by Carnegie Mellon University and sponsored by the Department
of Defense. Its objective is to provide leadership in software
engineering and in the transition of new software engineering
technology into practice.
[34] The CMMI ranks organizational maturity according to five levels.
Maturity levels 2 through 5 require verifiable existence and use of
certain key process areas. CMMI level 2, known as the "managed" level
is characterized by processes that, among other things are planned and
executed in accordance with policy; provide adequate resources; ensure
the designated work products are under the appropriate levels of
configuration management; involve relevant stakeholders; and are
monitored, controlled, and reviewed.
[35] [hyperlink, http://www.gao.gov/products/GAO-05-774].
[36] IRS, Applications Development: 5-Year Plan, "A systems view"
Projects-Programs-Retirements & Consolidations.
[37] GAO, Taxpayer Account Strategy: IRS Should Finish Defining
Benefits and Improve Cost Estimates, [hyperlink,
http://www.gao.gov/products/GAO-11-168] (Washington, D.C.: Mar. 24,
2011).
[38] GAO, Business Systems Modernization: Internal Revenue Service's
Fiscal Year 2007 Expenditure Plan, [hyperlink,
http://www.gao.gov/products/GAO-07-247] (Washington, D.C.: Feb. 15,
2007).
[39] Earned value management is a project management tool that
integrates the investment scope of work with schedule and cost
elements for investment planning and control. This method compares the
value of work accomplished during a given period with that of the work
expected in the period. Differences between accomplishments and
expectations are measured in both cost and schedule variances.
[40] In December 2007, the Associate Chief Information Officer for
Applications Development stated that IRS uses earned value management
to measure progress in completing enterprise life cycle deliverables,
not in delivering functionality, and consequently the agency's
application of earned value management does not provide a quantitative
measure of progress in meeting scope expectations.
[41] GAO, Business Systems Modernization: Internal Revenue Service's
Fiscal Year 2010 Expenditure Plan, [hyperlink,
http://www.gao.gov/products/GAO-10-539] (Washington, D.C.: May 10,
2010).
[42] [hyperlink, http://www.gao.gov/products/GAO-10-539].
[43] [hyperlink, http://www.gao.gov/products/GAO-10-539].
[44] [hyperlink, http://www.gao.gov/products/GAO-10-539].
[45] IRS's guidance defines milestones as management decision points
reviewing updated cost, progress, and risk information to make
continuation decisions. BSM programs undergo the following milestones:
Milestone 1-project initiation (vision and strategy), 2-conceptual
design or business architecture, 3-system logical design/system
architecture, 4-development, test, and integration, 5-system
deployment and transition to support and operations. The development,
test, and integration phase is sometimes split into two phases, each
corresponding to a different milestone: 4a-completion of system
physical design and 4b-development, test, and integration.
[46] IRS has not requested fiscal year 2011 funds for the AMS program
given that, according to IRS, as of fiscal year 2010 IRS began
incorporating all future AMS releases into the CADE 2 strategy.
[47] [hyperlink, http://www.gao.gov/products/GAO-10-539].
[48] We calculated the amount requested for LOE work by summing up the
requested amounts for all activities identified as LOE in the
expenditure plan.
[49] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009).
[50] Discrete work is work that has definite end products or events.
[51] [hyperlink, http://www.gao.gov/products/GAO-11-168].
[52] The new cycle will shift the weekly business processing cycle by
one day, and accelerated Individual Master File is the corresponding
one-day shift in the system processing cycle, which according to IRS,
leads to "accelerated" business benefits, such as faster returns.
[53] According to IRS, this amount represents an increase of $132
million from the preliminary estimate issued in July 2009. The
preliminary estimate, however, was a rough-order-of-magnitude estimate
that was not informed by the detailed design information for the
Database Implementation and Daily Processing projects that has been
developed since.
[54] GAO, Information Security: IRS Needs to Enhance Internal Control
over Financial Reporting and Taxpayer Data, [hyperlink,
http://www.gao.gov/products/GAO-11-308] (Washington, D.C.: Mar. 15,
2011).
[55] TIGTA, Annual Assessment of the Business System Modernization
Program, 2010-20-094 (Washington, D.C.: Sept. 23, 2010).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
