Information Technology
Significant Problems of Critical Automation Program Contribute to Risks Facing 2010 Census Gao ID: GAO-08-550T March 5, 2008Of the $11 billion total estimated cost of the 2010 Census, the Census Bureau planned (as of 2007) to spend about $3 billion on automation and information technology in order to improve census coverage, accuracy, and efficiency. Among other things, the Bureau is planning to automate many of its planned field data collection activities as a way to reduce costs and improve data quality and operational efficiency. Known as Field Data Collection Automation (FDCA), this initiative is acquiring handheld mobile computing devices that, along with other technology, are undergoing operational testing during a Census "Dress Rehearsal" that is taking place from February 2006 to June 2009. The $600 million FDCA program is a crucial component of the reengineered processes envisioned for the 2010 Census. In March 2006, GAO reported on the management capabilities of the FDCA program, and in October 2007, it reported on FDCA's status and plans. As requested, this testimony summarizes key findings from these two reports, as well as subsequent Bureau actions.
In March 2006, GAO reported that the FDCA project office had not implemented the full set of acquisition management capabilities (such as project and acquisition planning and requirements development and management) that were needed to effectively manage the program. For example, although the project office had developed baseline functional requirements for the acquisition, the Bureau had not yet validated and approved them. Valid baseline requirements are crucial for success, because they are the basis for the detailed operational requirements that determine system development. Without well-defined requirements, system acquisitions are at risk of cost increases, schedule delays, or performance shortfalls. Accordingly, GAO recommended that the Bureau validate and approve baseline requirements for FDCA. In October 2007, GAO reported that changes to requirements had been a contributing factor to both cost increases and schedule delays experienced by the FDCA program. Increases in the number of requirements led to the need for additional work and staffing. In addition, an initial underestimate of the contract costs caused both cost and schedule revisions. In response to the cost and schedule changes, the Bureau decided to delay certain system functionality, which increased the likelihood that the systems testing at the Dress Rehearsal would not be as comprehensive as planned. In addition, despite reports from Census field staff during spring Dress Rehearsal operations that data transmissions from handheld mobile computing devices were slow and inconsistent, the project team did not adequately identify performance issues with these devices as a risk. The magnitude of the performance difficulties was not clear, and the Bureau had not fully specified how it planned to measure the devices' performance, as GAO had recommended. Without effective management of these and other key risks, the FDCA program faced an increased probability that the system would not be delivered on schedule and within budget or perform as expected. Accordingly, GAO recommended that the FDCA project team strengthen its risk management activities, including risk identification and oversight. The Bureau has recently made efforts to further define the requirements for the FDCA program, and it has estimated that the revised requirements will result in significant cost increases. Rough estimates shared with the Congress and the Administration range from $600 million to $2 billion; however, specific programs and operations affected have not been identified, nor has the Bureau decided on a clear approach to address these issues. In view of the time frames for the 2010 Census, it is essential that the Bureau act rapidly to make decisions and to implement GAO's recommendations.
GAO-08-550T, Information Technology: Significant Problems of Critical Automation Program Contribute to Risks Facing 2010 Census
This is the accessible text file for GAO report number GAO-08-550T
entitled 'Information Technology: Significant Problems of Critical
Automation Program Contribute to Risks Facing 2010 Census' which was
released on March 5, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Testimony:
Before the Committee on Homeland Security and Governmental Affairs,
U.S. Senate:
For Release on Delivery:
Expected at 9:30 a.m. EST Wednesday, March 5, 2008:
Information Technology:
Significant Problems of Critical Automation Program Contribute to Risks
Facing 2010 Census:
Statement of David A. Powner:
Director, Information Technology Management Issues:
Mathew J. Scirè:
Director, Strategic Issues:
GAO-08-550T:
GAO Highlights:
Highlights of GAO-08-550T, a testimony before the Committee on Homeland
Security and Governmental Affairs, U.S. Senate.
Why GAO Did This Study:
Of the $11 billion total estimated cost of the 2010 Census, the Census
Bureau planned (as of 2007) to spend about $3 billion on automation and
information technology in order to improve census coverage, accuracy,
and efficiency. Among other things, the Bureau is planning to automate
many of its planned field data collection activities as a way to reduce
costs and improve data quality and operational efficiency. Known as
Field Data Collection Automation (FDCA), this initiative is acquiring
handheld mobile computing devices that, along with other technology,
are undergoing operational testing during a Census ……Dress Rehearsal‘‘
that is taking place from February 2006 to June 2009. The $600 million
FDCA program is a crucial component of the reengineered processes
envisioned for the 2010 Census. In March 2006, GAO reported on the
management capabilities of the FDCA program, and in October 2007, it
reported on FDCA‘s status and plans. As requested, this testimony
summarizes key findings from these two reports, as well as subsequent
Bureau actions.
What GAO Found:
In March 2006, GAO reported that the FDCA project office had not
implemented the full set of acquisition management capabilities (such
as project and acquisition planning and requirements development and
management) that were needed to effectively manage the program. For
example, although the project office had developed baseline functional
requirements for the acquisition, the Bureau had not yet validated and
approved them. Valid baseline requirements are crucial for success,
because they are the basis for the detailed operational requirements
that determine system development. Without well-defined requirements,
system acquisitions are at risk of cost increases, schedule delays, or
performance shortfalls. Accordingly, GAO recommended that the Bureau
validate and approve baseline requirements for FDCA. In October 2007,
GAO reported that changes to requirements had been a contributing
factor to both cost increases and schedule delays experienced by the
FDCA program. Increases in the number of requirements led to the need
for additional work and staffing. In addition, an initial underestimate
of the contract costs caused both cost and schedule revisions. In
response to the cost and schedule changes, the Bureau decided to delay
certain system functionality, which increased the likelihood that the
systems testing at the Dress Rehearsal would not be as comprehensive as
planned. In addition, despite reports from Census field staff during
spring Dress Rehearsal operations that data transmissions from handheld
mobile computing devices were slow and inconsistent, the project team
did not adequately identify performance issues with these devices as a
risk. The magnitude of the performance difficulties was not clear, and
the Bureau had not fully specified how it planned to measure the
devices‘ performance, as GAO had recommended. Without effective
management of these and other key risks, the FDCA program faced an
increased probability that the system would not be delivered on
schedule and within budget or perform as expected. Accordingly, GAO
recommended that the FDCA project team strengthen its risk management
activities, including risk identification and oversight. The Bureau has
recently made efforts to further define the requirements for the FDCA
program, and it has estimated that the revised requirements will result
in significant cost increases. Rough estimates shared with the Congress
and the Administration range from $600 million to $2 billion; however,
specific programs and operations affected have not been identified, nor
has the Bureau decided on a clear approach to address these issues. In
view of the time frames for the 2010 Census, it is essential that the
Bureau act rapidly to make decisions and to implement GAO‘s
recommendations.
What GAO Recommends:
In its reports, GAO made several recommendations to improve acquisition
management capabilities, including approving baseline requirements for
FDCA and strengthening risk management. The Bureau generally agreed
with them, but it has not yet fully implemented these and other
recommendations.
2010 Census:
Automation Problems and Uncertain Costs and Plans May Jeopardize the
Success of the Decennial and Warrant Immediate Attention:
Why GAO Assigns High Risk:
GAO audits and evaluations identify federal programs and operations
that are high risk due to their greater vulnerabilities to fraud,
waste, abuse, and mismanagement. Increasingly, GAO also is identifying
high-risk areas to focus on the need for broad-based transformations to
address major economy, efficiency, or effectiveness challenges. In
January 2007, GAO presented the 110th Congress with the latest status
of existing and new high-risk areas warranting attention by both the
Congress and the administration. Lasting solutions to high-risk
problems could save billions of dollars, dramatically improve service
to the public, strengthen confidence and trust in the performance and
accountability of the U.S. government, and ensure the ability of
government to deliver on its promises. The decennial census is a
Constitutionally-mandated activity that produces critical data used to
apportion congressional seats and allocate billions of dollars in
federal assistance.
Designation as High Risk:
Today GAO is designating the 2010 Census as a high-risk area. The
Decennial Census is at a critical stage in the 2008 Dress Rehearsal, in
which the Bureau has the opportunity to test, for the last time under
census-like conditions, its plans for 2010. GAO‘s work and Census have
highlighted a number of long-standing and emerging challenges facing
the 2010 Census. Because the census is one of the foundations for many
government decisions, threats to a timely and reliable census can
affect the public‘s confidence in government. GAO has reported long-
standing weaknesses in the Bureau‘s management of information
technology. To control costs and improve accuracy, the Bureau is
relying as never before on contractor provided technology, including
the use of handheld mobile computing devices. In October 2007, GAO
concluded that without effective management of key risks, the Field
Data Collection Automation (FDCA) program responsible for the devices
faced an increased probability that the system would not be delivered
on schedule and within budget or perform as expected. GAO recommended
that the Bureau strengthen its system testing and risk management
activities. Today the Bureau and its contractor are finalizing the FDCA
program and have not yet agreed on requirements for FDCA, and system
interface testing has been delayed. In the spring of 2007, the Bureau
tested the device under census-like conditions for the first time as
part of its address canvassing dress rehearsal. Field staff reported
technical problems with the handheld devices. The magnitude of these
problems is not clear and the Bureau still has not fully specified how
it will measure the performance of the handheld devices, as GAO
recommended. In addition, the Bureau has not performed recommended
analysis or provided sufficient information to provide a level of
confidence in its $11.5 billion life-cycle cost estimate of the
decennial census. The Bureau has not itemized the estimated costs of
each component operation, conducted sensitivity analysis on cost
drivers, or provided an explanation of significant changes in the
assumptions on which these costs are based. More recently, the Bureau
has delayed the dress rehearsal and dropped several operations.
Together, these weaknesses and actions raise serious questions about
the Bureau‘s preparations for conducting the 2010 Census. In 2005 GAO
highlighted the 2010 Census as an emerging area for possible high-risk
designation. GAO has recommended numerous corrective actions to address
the risks associated with the 2010 Census, but many of them have not
been implemented. GAO recommended that the Bureau (1) strengthen its
systems testing and risk management activities, (2) define specific
measurable performance requirements for the handheld mobile computing
device, and (3) develop an integrated and comprehensive plan to control
its costs and manage operations.
What GAO Recommends:
In its reports, GAO has recommended that the Bureau improve acquisition
management capabilities, operational planning, cost estimation, and
performance measurement. The Bureau agreed with most of these
recommendations, but has not fully implemented them.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.GAO-08-550T]. For more information, contact
Mathew Scire, 202-512-6806, sciremj@gao.gov or David A. Powner, 202-512-
9286, pownerd@gao.gov.
[End of section]
Mr. Chairman and Members of the Committee:
Thank you for the opportunity to participate in today's hearing to
discuss our past work on the Field Data Collection Automation (FDCA)
program--a key information technology acquisition that includes
systems, equipment, and infrastructure for field staff to use in
collecting census data for the 2010 Decennial Census. As you know, the
decennial census is mandated by the U.S. Constitution and provides data
that are vital to the nation. These data are used to apportion the
seats of the U.S. House of Representatives; realign the boundaries of
the legislative districts of each state; allocate billions of dollars
in federal financial assistance; and provide social, demographic, and
economic profiles of the nation's people to guide policy decisions at
each level of government.
Carrying out the census is the responsibility of the Department of
Commerce's Census Bureau, which is now preparing for the 2010 Census.
The next decennial census date is April 1, 2010, and the Secretary of
Commerce is required to report to the President the tabulation of total
population by states by December 31.[Footnote 1]
The Bureau plans to rely on automation and technology to improve the
coverage, accuracy, and efficiency of the 2010 Census. The Bureau
estimated that its information technology (IT) acquisitions would
account for about $3 billion of the currently estimated total $11.5
billion cost of the entire census. It is also holding what it refers to
as a Dress Rehearsal, a period centering around a mock Census Day on
May 1, 2008. Planned Dress Rehearsal activities include operational
testing of the 2010 Census systems in a census-like environment.
The Census Bureau has recently indicated that the FDCA program is
likely to incur significant cost overruns. As requested, our testimony
today will summarize our past reports on the FDCA program,[Footnote 2]
including our observations on the performance of the FDCA handheld
mobile computing devices, which were used during the April 2007 address
canvassing activities for the Dress Rehearsal, as well as subsequent
Bureau actions. In addition, we will discuss our designation of the
2010 Census as a high-risk program.
The work on which this testimony is based was performed in accordance
with generally accepted government auditing standards. Those standards
require that we plan and perform the audit to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and
conclusions based on our audit objectives. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Results in Brief:
We have previously reported on weaknesses in FDCA's IT acquisition
management activities.
* In March 2006, we reported that the FDCA project office had not
implemented the full set of acquisition management capabilities (such
as project and acquisition planning, and requirements development and
management) that were needed to effectively manage the
program.[Footnote 3] Specifically, the FDCA project had completed most
of its solicitation activities, but activities in other management
areas had been initiated but not completed. Further, although the
project office had developed baseline functional requirements for the
acquisition, the Bureau had not yet validated and approved them. Valid
baseline requirements are crucial for success, because they are the
basis for the detailed operational requirements that determine system
development. Without well-defined requirements, system acquisitions are
at risk of cost increases, schedule delays, or performance shortfalls.
Finally, the project office had identified risks, but it had not yet
implemented a risk management process. Until these and other basic
management activities were fully implemented, the project faced
increased risks that the system would experience cost overruns,
schedule delays, and performance shortfalls. We made recommendations to
the Director of the Census Bureau to ensure that the project office
completed key activities needed to effectively manage its acquisitions,
including activities associated with effective requirements management.
* In October 2007, we reported that changes to requirements had been a
contributing factor to both schedule delays and cost increases
experienced by the FDCA program.[Footnote 4] The schedule had been
revised--resulting in delays in system development and testing of
interfaces--and the life-cycle cost estimates for this program had
increased to $647 million, with additional cost increases anticipated.
According to the contractor, the overrun was due primarily to an
increase in the number of system requirements. We also reported that
the FDCA project team had not adequately identified risks associated
with performance issues experienced by handheld mobile computing
devices. In May and June 2007, both we and the Census Bureau had
observed the use of the handheld mobile computing device in Census-like
conditions, which revealed a number of performance issues with the
devices, such as slow and inconsistent data processing. The magnitude
of these performance issues was not clear. If the project team did not
implement key risk management activities, it would face an increased
probability that the system would not be delivered on schedule and
within budget or perform as expected. To help ensure that the program
operates as intended, we recommended that the project team strengthen
risk management activities, including those associated with risk
identification, mitigation, and oversight. In written comments on a
draft of both reports, the department generally agreed with our
recommendations for the FDCA project.
The Bureau has recently made efforts to further define the requirements
for the FDCA program, and it has estimated that the revised
requirements will result in significant cost increases. Rough estimates
that have been shared with the Congress and the Administration range
from $600 million to $2 billion; however, specific programs and
operations affected have not been identified, nor has the Bureau
decided on a clear approach to address these issues. In view of the
timeframes for the 2010 Census, it is essential that the Bureau act
rapidly to make decisions and to implement our previous
recommendations.
In 2005 we highlighted the 2010 Census as an emerging area for possible
high-risk designation. In addition to the recommendations discussed
above, we have recommended other corrective actions to address the
risks associated with the 2010 Census, but many of these have not yet
been fully implemented. Among these are recommendations that the Bureau
strengthen its systems testing, define specific measurable performance
requirements for the handheld mobile computing device, and develop an
integrated and comprehensive plan to control its costs and manage
operations.[Footnote 5] More recently, the Bureau has delayed the Dress
Rehearsal and dropped several operations, some of which are designed to
reach historically undercounted populations and measure census
coverage. Together, these concerns raise serious questions about the
Bureau's preparations for conducting the 2010 Census. Because the
census is one of the foundations for many government decisions, threats
to a timely and reliable census can affect the public's confidence in
government. We are, therefore, today designating the 2010 Census as
high risk.
Background:
Conducting the decennial census is a major undertaking involving many
interrelated steps, including:
* identifying and correcting addresses for all known living quarters in
the United States (known as "address canvassing");
* sending questionnaires to housing units;
* following up with nonrespondents through personal interviews;
* identifying people with nontraditional living arrangements;
* managing a voluminous workforce responsible for follow-up activities;
* collecting census data by means of questionnaires, calls, and
personal interviews;
* tabulating and summarizing census data; and:
* disseminating census analytical results to the public.
The decennial census program is the responsibility of the bureau's
Decennial Census organization, which is made up of several divisions
and offices. In particular, the Decennial Management Division is
responsible for implementing the decennial census. The Decennial
Systems and Contracts Management Office manages selected system
contracts supporting the decennial census including FDCA.
FDCA Plays Crucial Role in the Decennial Census:
The Bureau estimated that it would spend about $3 billion on automation
and IT for the 2010 Census to improve coverage, accuracy, and
efficiency. Figure 1 shows the key systems and interfaces supporting
the 2010 Census.
Figure 1: Key 2010 Census Systems and Interfaces:
This figure is a flowchart showing key 2010 census systems and
interfaces.
[See PDF for image]
Source: U.S. Census Bureau.
Note: Shaded boxes indicate major IT decennial systems.
[End of figure]
To collect respondent information (a process depicted in the middle
section of fig. 1), the Bureau is pursuing the FDCA program. FDCA is
expected to provide automation support for field data collection
operations, as well as reduce costs and improve data quality and
operational efficiency. As the figure shows, the FDCA technology is
central to the reengineered processes that are envisioned for the 2010
Census. The acquisition includes the systems, equipment, and
infrastructure that field staff will use to collect census data,
including handheld mobile computing devices.[Footnote 6] The program is
expected to provide office automation for regional and local census
offices; the telecommunications infrastructure for headquarters,
regional, and local offices; and mobile computing devices for field
workers.
The bureau plans to have field-based enumerators use nearly 500,000
mobile computing devices to support decennial census field operations.
Enumerators from local census offices will use these mobile computing
devices to complete activities including address canvassing (visiting
households to update address lists and collect global positioning
coordinates to update maps) and conducting nonresponse follow-up
(visiting households to obtain information from households that do not
provide responses via mail, Internet, or phone).[Footnote 7]
In preparation for the 2010 Census, the Bureau plans a series of tests
of its (new and existing) operations and systems in different
environments, as well as to conduct what it refers to as the Dress
Rehearsal. During the Dress Rehearsal period, which runs from February
2006 through June 2009, the Bureau plans to conduct development and
testing of systems, run a mock Census Day, and prepare for Census 2010,
which will include opening offices and hiring staff. These Dress
Rehearsal activities are to provide an operational test of the
available system functionalities in a census-like environment, as well
as other operational and procedural activities.
Previously Reported Weaknesses in Acquisition Management Activities
Increase Risks to Census:
In previous reports, we have described weaknesses in FDCA's IT
acquisition management activities that pose serious risks to the
success of the 2010 Census. In March 2006, we testified that the FDCA
project office had not implemented the full set of acquisition
management capabilities that were needed to effectively manage the
program; we noted in particular that although baseline functional
requirements had been developed, they had not been validated and
approved.[Footnote 8] Subsequently, in October 2007, we reported that
changes to requirements had been a contributing factor to both schedule
delays and cost increases experienced by the FDCA program.[Footnote 9]
Further, although the FDCA project team had performed many practices
associated with establishing sound and capable risk management
processes, critical weaknesses remained. For example, despite reports
of performance issues with handheld mobile computing devices, the
project team did not adequately identify these as a risk.
The FDCA Program Had Not Initiated Key Management Activities:
In March 2006, we reported that the FDCA project office had initiated
many key processes to oversee and manage the program, but had not yet
implemented the full set of acquisition management capabilities it
needed to fully manage the acquisition. Table 1 provides a summary of
the status of the FDCA project in each of the process areas we
evaluated.
Table 1: Summary of the Status of FDCA Acquisition Management
Capabilities as of March 2006:
Capability: Project and acquisition planning;
Status: Initiated, but key activities remained to be completed, such as
identifying deliverables and milestones, and obtaining stakeholder buy-
in on a project plan that included the project's estimated costs,
budget, and schedules.
Capability: Solicitation;
Status: Key activities completed; bureau officials said that they were
on schedule to award the contract in March 2006.
Capability: Requirements development and management; Status: Initiated,
but key activities remained to be completed, such as validating
requirements.
Capability: Risk management;
Status: Initiated, but key activities remained to be performed, such as
assigning responsibilities for risks and preparing mitigation plans.
Capability: Contract tracking and oversight/project monitoring and
control;
Status: Initiated, but key performance measures had not yet been
established; bureau officials said that they expected to define these
metrics after contract award.
Capability: Process and product quality assurance;
Status: Initiated; the quality assurance process was expected to be
implemented after contract award.
Capability: Executive oversight/governance;
Status: Initiated, but key governance activities remained to be
completed.
Source: GAO analysis.
[End of table]
The FDCA project office had made some progress in building management
capabilities, but more remained to be done in key areas. For example,
the project office completed many key solicitation activities and
planned to award a contract in March 2006. In the requirements
development and management area, the project office and the Decennial
Management Division developed some FDCA-specific functional
requirements with participation from stakeholders. The office had also
drafted a requirements management plan. However, the division had not
yet validated and approved a baseline set of operational requirements
nor ensured traceability between its operational requirements and the
FDCA request for proposals. Until the bureau finalized its operational
requirements for the census and ensured that the FDCA request for
proposal was consistent with the baseline requirements, the project
would be at risk of having changes to the requirements, potentially
affecting its ambitious development and implementation schedule.
Additionally, in the contract and project monitoring area, the project
office had initiated efforts to oversee the contractor's performance,
such as requiring earned value management reporting and hiring staff
with contract tracking and oversight experience. However, the project
office had not yet selected detailed performance measures for tracking
the contractor or its own internal progress (such as progress against
planned milestones, number of risks opened and closed, number and
frequency of changes to requirements, and frequency of quality
assurance reviews). Without such practices in place, the project team
would not be able to ensure that it could identify problems and take
appropriate corrective actions in a timely manner.
We said that sound acquisition and management processes were needed to
minimize risks and thereby maximize chances for success. Such processes
have been identified and endorsed by such leading organizations as the
Software Engineering Institute and the Chief Information Officers'
Council, as well as in our prior work analyzing best practices in
industry and government. Our work has shown that such processes are
significant factors in successful systems acquisitions and development
programs, and they improve the likelihood of meeting cost and schedule
estimates as well as performance.
To ensure that the bureau adequately managed the FDCA project, we
recommended that the Director of the Census Bureau direct the project
office to take four actions, including validating and approving
baseline requirements. The Bureau generally agreed with our
recommendations and stated that time constraints and budgets had driven
it to proceed with its acquisitions before all the recommended
activities had been completed.
The FDCA Program Had Delayed Functionality and Was Experiencing Cost
Increases:
As of October 2007, we reported that FDCA had experienced schedule
delays and cost increases. Specifically, the life cycle cost estimates
for this program had increased to $647 million, and additional cost
increases were anticipated. In response to schedule delays as well as
other factors, including cost, the Bureau made schedule adjustments and
planned to delay certain system functionality. As a result of the FDCA
schedule changes, the likelihood had increased that the systems testing
at the Dress Rehearsal would not be as comprehensive as planned.
In October 2007, we reported that the contractor was in the process of
developing and testing FDCA software for the Dress Rehearsal Census
Day, and had delivered 1,388 handheld mobile computing devices to be
used in address canvassing[Footnote 10] for the Dress Rehearsal. Also,
key FDCA support infrastructure had been installed, including the
Security Operation Center. In future contract phases, the project was
to continue development, deploy systems and hardware, support census
operations, and perform operational and contract closeout activities.
However, the Bureau revised FDCA's original schedule and delayed or
eliminated some of its key functionality from the Dress Rehearsal,
including the automated software distribution system. According to the
Bureau, it revised the schedule because it realized that it had
underestimated the costs for the early stages of the contract, and that
it could not meet the contractor's estimated level of first-year
funding because the fiscal year 2006 budget was already in place.
According to the Bureau, this initial underestimate led to schedule
changes and overall cost increases.
In particular, the project life-cycle costs had increased. At contract
award in March 2006, the total cost of FDCA was estimated not to exceed
$596 million. As of September 2006, the cost estimate had increased to
$624 million. In May 2007, the life-cycle cost rose by a further $23
million because of increasing system requirements, which resulted in an
estimated life-cycle cost of about $647 million. Table 2 shows the life-
cycle cost estimates for FDCA as of October 2007.
Table 2: FDCA Life-Cycle Cost Estimates as of October 2007:
Execution period: Baseline planning period;
Start date: March 31, 2006;
End date: June 30, 2006;
Cost estimates (in millions): September 2006: $11;
Cost estimates (in millions): May 2007: $11.
Execution period: Execution Period 1;
Start date: July 1, 2006;
End date: December 31, 2008;
Cost estimates (in millions): September 2006: 200;
Cost estimates (in millions): May 2007: 225.
Execution period: Execution Period 2;
Start date: January 1, 2009;
End date: September 30, 2011;
Cost estimates (in millions): September 2006: 319;
Cost estimates (in millions): May 2007: 318.
Execution period: Execution Period 3;
Start date: August 1, 2010;
End date: End of contract;
Cost estimates (in millions): September 2006: 10;
Cost estimates (in millions): May 2007: 10.
Execution period: Leased equipment;
Start date: N/A;
End date: N/A;
Cost estimates (in millions): September 2006: 12;
Cost estimates (in millions): May 2007: 12.
Execution period: Management reserve;
Start date: N/A;
End date: N/A;
Cost estimates (in millions): September 2006: 7;
Cost estimates (in millions): May 2007: 5.
Execution period: Award fee;
Start date: N/A;
End date: N/A;
Cost estimates (in millions): September 2006: 65;
Cost estimates (in millions): May 2007: 65.
Execution period: Total;
Start date: [Empty];
End date: [Empty];
Cost estimates (in millions): September 2006: $624;
Cost estimates (in millions): May 2007: $647.
Source: GAO analysis of Census Bureau data.
Note: Total may not add due to rounding.
[End of table]
In addition, FDCA had already experienced $6 million in cost overruns,
and both our analysis and the contractor's analysis expected FDCA to
experience additional cost overruns. Based on our analysis of cost
performance reports (from July 2006 to May 2007), we projected that the
FDCA project would experience further cost overruns by December 2008.
According to the contractor, the major cause of projected cost overruns
was the system requirements definition process. For example, in
December 2006, the contractor noted a significant increase in the
requirements for the Dress Rehearsal Paper Based Operations in
Execution Period 1. According to the cost performance reports, this
increase meant that more work had to be conducted and more staffing
assigned to meet the Dress Rehearsal schedule.
The Bureau agreed that cost increases occurred in some cases because of
the addition of new requirements, most of which related to the security
of IT systems, but added that in other cases, increases occurred from
the process of the contractor converting high-level functional
requirements into more detailed specific requirements. However, the
process of developing detailed requirements from high-level functional
requirements does not inevitably lead to cost increases if the
functional requirements were initially well-defined.
We reported that the FDCA schedule changes had increased the likelihood
that the systems testing at the Dress Rehearsal would not be as
comprehensive as planned. The inability to perform comprehensive
operational testing of all interrelated systems increases the risk that
further cost overruns will occur and that decennial systems will
experience performance shortfalls.
The FDCA Project Team Was Making Progress in Risk Management
Activities, but Critical Weaknesses Remained:
According to the Software Engineering Institute (SEI), the purpose of
risk management is to identify potential problems before they occur.
When problems are identified, risk-handling activities can be planned
and invoked as needed across the life of a project in order to mitigate
adverse impacts on objectives. Effective risk management involves early
and aggressive risk identification through the collaboration and
involvement of relevant stakeholders. Based on SEI's Capability
Maturity Model® Integration (CMMI®), risk management activities can be
divided into four key areas:
* preparing for risk management,
* identifying and analyzing risks,
* mitigating risks, and:
* executive oversight.
The discipline of risk management is important to help ensure that
projects are delivered on time, within budget, and with the promised
functionality. It is especially important for the 2010 Census, given
the established timeframe.
In October 2007, we reported that the FDCA project had made progress in
implementing risk management practices, but some weaknesses remained.
For example, the project team had developed strategies to identify the
scope of the risk management effort. However, the project team had
weaknesses in identifying risks, establishing adequate mitigation
plans, and reporting risk status to executive-level officials. These
weaknesses in completing key risk management activities can be
attributed in part to the absence of Bureau policies for managing major
acquisitions, as we described in earlier reports.[Footnote 11] Without
effective risk management practices, the likelihood of project success
is decreased.
Weaknesses in identifying risks. As of July 2007, the FDCA project team
had not identified or documented any significant risks related to the
handheld computers that will be used in the 2010 Census, despite
problems arising during the Dress Rehearsal. The computers are designed
to automate operations for field staff and eliminate the need to print
millions of paper questionnaires and maps used by temporary field staff
to conduct address canvassing and nonresponse follow-up. Automating
operations may allow the Bureau to reduce the cost of operations; thus,
it is critical that the risks surrounding the use of the handheld
computers be closely monitored and effectively managed to ensure their
success.
However, the Bureau had not identified or documented risks associated
with a variety of handheld computers' performance problems that we
identified through field work.[Footnote 12] Specifically, we found that
during Dress Rehearsal activities between May 2007 and June 2007, as
the Bureau tested a prototype of the handheld computers, field staff
experienced multiple problems. For example, the field staff told us
that they experienced slow and inconsistent data transmissions from the
handheld computers to the central data processing center. The field
staff reported that the device was slow to process addresses that were
a part of a large assignment area.[Footnote 13] Bureau staff reported
similar problems with the handheld computers in observation reports,
help desk calls, and debriefing reports. In addition, a time and motion
study conducted by the Census Bureau indicated that field staff
reported significant downtime in two test locations--about 23 percent
in one location and about 27 percent in another location. The study
also described occurrences of failed transmissions and field staff
attempts to resolve transmission problems.
Collectively, the observation reports, help desk calls, debriefing
reports, and time-and-motion study raised serious questions about the
performance of the handheld computers during the address canvassing
operation. According to the Bureau, the contractor used these
indicators to identify and address underlying problems during the Dress
Rehearsal. Still, the magnitude of the handheld computers' performance
issues throughout the Dress Rehearsal was not clear. For example, the
Bureau received analyses from the contractor on average transmission
times. However, the contractor had not provided analyses that show the
full range of transmission times, nor how this might have changed
throughout the whole operation.
In addition, the Bureau has not fully specified how it will measure
performance of the handheld computers, even though the FDCA contract
anticipates the Bureau's need for data on the performance of the
handheld computers. The FDCA program outlines the type of data the
contractor is to provide the Bureau on the performance of the handheld
computers. Specifically, sections of the FDCA contract require the
handheld computers to have a transmission log showing what was
transmitted, the date, time, user, destination, content/data type, and
the outcome status. Another section of the Bureau's FDCA contract
states that the FDCA contractor shall provide near real time reporting
and monitoring of performance metrics and a "control panel/dash board"
application to visually report those metrics from any Internet-enabled
computer. However, the contractor and the Bureau are not using a
dashboard for Dress Rehearsal activities. Rather, during the Dress
Rehearsal, the Bureau plans to identify what data and performance it
would need for tracking the performance of the handheld computers in
2010 operations.
In order for the Bureau to ensure that the FDCA handheld computers are
ready for full scale operations, it will have to be able to measure
performance. We recommended in a report on the Bureau's earlier version
of the handheld computers that the Bureau define specific, measurable
performance requirements for the handheld computer and other census-
taking activities that address such important measures as productivity,
cost savings, reliability, and durability, and that the Bureau test the
device's ability to meet those requirements in 2006.[Footnote 14] We
also recommended in a March 2006 testimony that the Bureau validate and
approve FDCA baseline requirements.[Footnote 15] To get these things
done, the Bureau must work within a time frame that has now been
compressed. By law, the decennial census date is April 1, 2010, and the
results must be submitted to the President in December 2010. Access to
real-time performance metrics via a "control panel/dash board" would
assist Bureau management in assessing the handheld computer's
performance and maximize the amount of time the Bureau and the
contractor would have to remedy any problems that are identified during
any operations.
Further, the Bureau's tight 2010 decennial operations schedule allows
little time for fixing problems with the device, raising the importance
of the Bureau's access to these performance indicators. Such data would
help fully inform stakeholders of the risks associated with the
handheld computer and allow project teams to develop mitigation
activities to help avoid, reduce, and control the probability of these
risks occurring.
Finally, the FDCA project team did not provide evidence that specific
system interface risks were being adequately identified to ensure that
risk handling activities would be invoked should the systems fail
during the 2010 Census. If risks are not adequately identified and
analyzed, management may be prevented from monitoring and tracking
risks and taking the appropriate mitigation actions, increasing the
probability that the risks will materialize and magnifying the extent
of damage incurred in such an event.
Weaknesses in establishing adequate mitigation plans. The FDCA project
team had developed mitigation plans that were often untimely or
included incomplete activities and milestones for addressing the risks.
For example, the FDCA project team had developed mitigation plans for
the most significant risks, but the plans did not always identify
milestones for implementing mitigation activities. Moreover, the plans
did not identify any commitment of resources, several did not establish
a period of performance, and the team did not always update the plans
with the latest information on the status of the risk. In addition, the
FDCA project team did not provide evidence of developing mitigation
plans to handle the other significant risks as described in its risk
mitigation strategy. (These risks included a lack of consistency in
requirements definition and insufficient FDCA project office staffing
levels).
Further, the project team did not always implement the mitigation plans
as appropriate. For example, the project team did not identify system
interface risks nor prepare adequate mitigation plans to ensure that
systems would operate as intended. Because they did not develop
complete mitigation plans, the project team could not ensure that for a
given risk, techniques and methods would be invoked to avoid, reduce,
and control the probability of occurrence.
Weaknesses in reporting risk status to executive-level officials.
Reviews of the project teams' risk management activities, status, and
results should be held on a periodic and event-driven basis. The
reviews should include appropriate levels of management, such as key
Bureau executives, who can provide visibility into the potential for
project risk exposure and appropriate corrective actions. The FDCA
project team did not provide sufficient evidence to document that the
status of risks was discussed with executive-level officials or what
the discussions covered. Failure to report a project's risks to
executive-level officials reduces the visibility of risks to executives
who should be playing a role in mitigating them.
To strengthen the FDCA program risk management activities, we
recommended that the Director of the Census Bureau ensure that the
project teams, including FDCA, implement actions to address weaknesses
in risk management. The Bureau generally agreed with our
recommendations, but disagreed with our assessment with regard to risk
identification, pointing out that the FDCA project identified risks
associated with handheld mobile computing devices and assigned
responsibility to the contractors. In addition, the Bureau said that it
had identified risks related to system interfaces. However, the project
did not identify significant risks for the project office to monitor
and track related to problems arising during the address canvassing
component of the Dress Rehearsal. Also, although this project
identified a general risk related to system interfaces, it did not
identify specific risks related to particular interfaces.
In June 2007, an independent assessment of the FDCA program reported on
requirements management problems similar to those we reported in March
2006. Specifically, the study reported that the contractor is
overwhelmed by a substantial increase in requirements, having thousands
of unreconciled (that is, not validated) requirements. According to the
study, reconciliation of requirements was impeded by many factors,
including the number of requirements and lack of agreement among
stakeholders on what to include in requirements. Similar to our
recommendation in the March 2006 report, the study recommended that the
Bureau immediately stabilize requirements by defining and refining
them.
The Bureau has recently made efforts to further define the requirements
for the FDCA program, and it has estimated that the revised
requirements will result in significant cost increases. Rough estimates
that have been shared with the Congress and the Administration range
from $600 million to $2 billion; however, specific programs and
operations affected have not been identified, nor has the Bureau
decided on a clear approach to address these issues.
2010 Census Is at Risk:
The Decennial Census is at a critical stage in the 2008 Dress
Rehearsal, in which the Bureau has the opportunity to test, for the
last time, its plans for 2010. Our work, over the years, and the Bureau
itself have highlighted a number of challenges facing the 2010 Census.
These include (as discussed above) weaknesses in the Bureau's IT
acquisition management capabilities and risks associated with the
performance of the handheld mobile computing device. In addition to
these challenges, concerns have been raised in two other areas: the
uncertainty over the ultimate cost of the 2010 Census and the
elimination of several operations from the 2008 Dress Rehearsal:
* The Bureau has not performed the analysis or provided sufficient
information to provide a level of confidence in its $11.5 billion life-
cycle cost estimate of the decennial census. The Bureau has not
itemized the estimated costs of each component operation, conducted a
sensitivity analysis on cost drivers, or provided an explanation of
significant changes in the assumptions on which these costs are based.
* The Dress Rehearsal, designed to be a dry run of the census, is the
Bureau's last opportunity to ensure that all the various operations and
systems are in sync before the 2010 Census. However, the Bureau has
delayed the dress rehearsal and dropped several operations, including
operations that are designed to reach populations that are typically
undercounted, measure census coverage, and enumerate people living in
rural areas. Although the Bureau has carried out these operations
before, in some cases they now involve new procedures and system
interfaces that, as a result of their exclusion, will not be tested
under census-like conditions. Further, end-to-end testing of the
various IT systems will not be performed under census-like conditions
because such testing will not occur during the Dress Rehearsal.
Given its size and complexity, the Census presents significant risks,
and implementing our recommendations would help the Bureau effectively
manage the myriad interrelated operations needed to ensure an accurate
and complete count in 2010. The success of this effort is put in
jeopardy by the risks discussed above. Managing these risks is critical
to the timely completion of a reliable and cost-effective census.
We have recommended numerous corrective actions to address the risks
associated with the 2010 Census, but many of them have not been
implemented. Besides those already discussed, in January 2004 we made
recommendations that the Bureau control the costs of the census and
develop an integrated and comprehensive plan to manage
operations.[Footnote 16] Specifically, to understand and manage the
assumptions that drive the cost of the Decennial Census, we asked the
Bureau to annually update the cost of the 2010 Census and conduct
sensitivity analysis on the $11.5 billion estimate. However, while the
Bureau understands the utility of sensitivity analysis, it has not
conducted such analysis. To effectively manage operations, we
recommended that the Bureau develop a comprehensive, integrated project
plan for the 2010 Census. In response, the Bureau developed a 2010
Census Operations and Systems Plan. This document includes inputs and
outputs and describes linkages among operations and systems. However,
it does not yet include a risk mitigation plan, a detailed 2010 Census
timeline, and the cost for each operation.
In 2005, we highlighted the 2010 Census as an emerging area to be
monitored for possible high-risk designation, and in 2006, we listed it
as a key oversight issue for the 110TH Congress.[Footnote 17] To
determine whether the 2010 Census is high risk, we have considered a
set of quantitative and qualitative factors.[Footnote 18] Those factors
relevant to the 2010 Census include whether the investment is large,
exceeding $1 billion; whether the risk is detrimental to service
delivery and citizens' rights; and whether the risk could result in
unreliable decision-making data and a reduced confidence in government.
Based on our work, the 2010 Census meets those criteria. As noted
above, the decennial census is a large investment, and there is a
potential risk for an incomplete or inaccurate census, which would be
detrimental to the administration of federal financial assistance for
government programs and to the enforcement, for example, of citizens'
rights under the Voting Rights Act. Furthermore, because the census is
a foundation for elected government, threats to a timely and reliable
census can affect the public's confidence in government. We are
therefore designating the 2010 Census as high risk.
In summary, the FDCA program was expected to support the goal of a
reengineered 2010 Decennial Census to reduce costs and improve data
quality and operational efficiency. Because the Bureau did not validate
and approve the FDCA program requirements, it faces a crisis, including
increased costs and schedule delays. In this emergency, the Bureau will
have to make quick decisions to ensure that the project can achieve its
purpose. Continued attention by this Committee, Commerce, the Bureau,
us, and others will be essential to helping ensure a cost-effective and
efficient 2010 census.
Mr. Chairman and members of the committee, this concludes our
statement. We would be happy to respond to any questions that you or
members of the committee may have at this time.
Contacts and Staff Acknowledgments:
If you have any questions on matters discussed in this testimony,
please contact David A. Powner at (202) 512-9286 or Mathew Scirè at
(202) 512-6806 or by e-mail at pownerd@gao.gov or sciremj@gao.gov.
Other key contributors to this testimony include Mathew Bader, Thomas
Beall, Barbara Collier, Jeffrey DeMarco, Elizabeth Hosler, Richard
Hung, Franklin Jackson, Barbara Lancaster, Andrea Levine, Signora May,
Lisa Pearson, Cynthia Scott, Niti Tandon, Jonathan Ticehurst, and
Timothy Wexler.
Footnotes:
[1] 13 U.S.C. §§141(a) and (b).
[2] GAO, Census Bureau: Important Activities for Improving Management
of Key 2010 Decennial Acquisitions Remain to be Done, GAO-06-444T
(Washington, D.C.: Mar. 1, 2006) and Information Technology: Census
Bureau Needs to Improve Its Risk Management of Decennial Systems, GAO-
08-79 (Washington, D.C.: Oct. 5, 2007).
[3] We also reported on the Decennial Response Integration System,
which is to provide a system for collecting and integrating census
responses from all sources, including forms, telephone interviews, and
handheld computing devices in the field.
[4] GAO-08-79.
[5] GAO, 2010 Census: Cost and Design Issues Need to be Addressed Soon,
GAO-04-37 (Washington, D.C.: Jan 15, 2004) and 2010 Census: Basic
Design Has Potential But Remaining Challenges Need Prompt Resolution,
GAO-05-9 (Washington, D.C. : Jan. 12, 2005).
[6] Handheld mobile computing devices will be used to update the
Bureau's address list, to perform follow-up at addresses for which no
questionnaire was returned, and to perform activities to measure census
coverage.
[7] Further details of mobile computing devices are provided in GAO,
2010 Census: Planning and Testing Activities Are Making Progress, GAO-
06-465T (Washington, D.C.: Mar. 1, 2006).
[8] GAO-06-444T.
[9] GAO-08-79.
[10] Address canvassing is a field operation to build a complete and
accurate address list. In this operation, census field workers go door
to door verifying and correcting addresses for all households and
street features contained on decennial maps.
[11] GAO, Information Technology Management: Census Bureau Has
Implemented Many Key Practices, but Additional Actions Are Needed, GAO-
05-661 (Washington, D.C.: June 16, 2005) and Census Bureau: Important
Activities for Improving Management of Key 2010 Decennial Acquisitions
Remain to be Done, GAO-06-444T (Washington, D.C.: Mar. 1, 2006).
[12] We testified on the preliminary results from this field work, as
well as the results of our October report, in GAO, Information
Technology: Census Bureau Needs to Improve Its Risk Management of
Decennial Systems, GAO-08-259T (Washington, D.C.: Dec. 11, 2007).
[13] A small geographic area, usually a block or group of blocks,
established by the Census Bureau as a basic unit for data collection by
a single enumerator, lister, or other field staff.
[14] GAO, 2010 Census: Basic Design Has Potential, but Remaining
Challenges Need Prompt Resolution, GAO-05-9 (Washington, D.C.: Jan.12,
2005).
[15] GAO-06-444T.
[16] GAO, 2010 Census: Cost and Design Issues Need to Be Addressed
Soon, GAO-04-37 (Washington, D.C.: Jan. 15, 2004).
[17] GAO, Suggested Areas for Oversight for the 110th Congress, GAO-07-
235R (Washington, D.C.: November 2006).
[18] GAO, Determining Performance and Accountability Challenges and
High Risks, GAO-01-159SP (Washington, D.C.: November 2007).
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
