2010 Census
Census Bureau Continues to Make Progress in Mitigating Risks to a Successful Enumeration, but Still Faces Various Challenges
Gao ID: GAO-10-132T October 7, 2009
The decennial census is a constitutionally-mandated activity that produces data used to apportion congressional seats, redraw congressional districts, and help allocate billions of dollars in federal assistance. In March 2008, GAO designated the 2010 Census a high-risk area in part because of information technology (IT) shortcomings. The U.S. Census Bureau (Bureau) has since strengthened its risk management efforts and made other improvements; however, in March 2009, GAO noted that a number of challenges and uncertainties remained. This testimony discusses the Bureau's readiness for 2010 and covers: (1) the delivery of key IT systems, (2) preliminary findings on the results of address canvassing and the lessons learned from that operation that can be applied to subsequent field operations, and (3) the Bureau's progress in improving its cost estimation abilities. The testimony is based on previously issued and ongoing GAO work.
The Bureau continues to make noteworthy gains in mitigating risks and in keeping the headcount on-track, but a number of challenges remain. Specifically, over the last few months, the Bureau has made important strides in improving oversight of testing key IT systems. For example, the Bureau named a testing officer to monitor the testing of census-taking activities. The Bureau has also made progress in system testing, but faces tight timeframes in finalizing the paper-based operations control system (PBOCS), which will be used to manage field operations. If any significant problems are identified during the testing phases of PBOCS, there will be little time, in most cases, to resolve the problems before the system needs to be deployed. Address canvassing, an operation where temporary workers known as listers go door-to-door to verify and update address data, finished ahead of schedule, but was over budget. Based on initial Bureau data, the preliminary figure on the actual cost of address canvassing is $88 million higher than the original estimate of $356 million, an overrun of 25 percent. A key reason for the overrun is that the Bureau did not update its cost estimates to reflect the changes to the address canvassing workload. Further, the Bureau did not follow its staffing strategy and hired too many listers. The Bureau's efforts to fingerprint employees, which was required as part of a criminal background check, did not proceed smoothly, in part because of training issues. As a result, over 35,000 temporary census workers--over a fifth of the address canvassing workforce--were hired despite the fact that their fingerprints could not be processed and they were not fully screened for employment eligibility. The Bureau is refining instruction manuals and taking other steps to improve the fingerprinting process for future operations. GAO is unable to verify the accuracy of the $14.7 billion estimated cost of the 2010 Census because key details and assumptions are unavailable. However, the Bureau is taking steps to improve its cost estimation process for 2020, including training its staff in cost estimation skills. While the Bureau has taken a number of actions to mitigate risk and its overall readiness for 2010 has improved, much work remains to be done. Many things can happen over the next few months, and keeping the entire enterprise on-plan will continue to be a daunting challenge fraught with risks. High levels of public participation, and continued Bureau and congressional attention to stewardship, performance, and accountability, will be key to a successful census.
GAO-10-132T, 2010 Census: Census Bureau Continues to Make Progress in Mitigating Risks to a Successful Enumeration, but Still Faces Various Challenges
This is the accessible text file for GAO report number GAO-10-132T
entitled '2010 Census: Census Bureau Continues to Make Progress in
Mitigating Risks to a Successful Enumeration, but Still Faces Various
Challenges' which was released on October 7, 2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Federal Financial Management, Government
Information, Federal Services and International Security, Committee on
Homeland Security and Government Affairs, U.S. Senate:
United States Government Accountability Office:
GAO:
For Release on Delivery:
Expected at 3:00 p.m. EDT:
Wednesday, October 7, 2009:
2010 Census:
Census Bureau Continues to Make Progress in Mitigating Risks to a
Successful Enumeration, but Still Faces Various Challenges:
Statement of Robert Goldenkoff:
Director, Strategic Issues:
GAO-10-132T:
GAO Highlights:
Highlights of GAO-10-132T, a testimony before the Subcommittee on
Federal Financial Management, Government Information, Federal Services,
and International Security, Committee on Homeland Security and
Governmental Affairs, U.S. Senate.
Why GAO Did This Study:
The decennial census is a constitutionally-mandated activity that
produces data used to apportion congressional seats, redraw
congressional districts, and help allocate billions of dollars in
federal assistance. In March 2008, GAO designated the 2010 Census a
high-risk area in part because of information technology (IT)
shortcomings. The U.S. Census Bureau (Bureau) has since strengthened
its risk management efforts and made other improvements; however, in
March 2009, GAO noted that a number of challenges and uncertainties
remained. This testimony discusses the Bureau‘s readiness for 2010 and
covers: (1) the delivery of key IT systems, (2) preliminary findings on
the results of address canvassing and the lessons learned from that
operation that can be applied to subsequent field operations, and (3)
the Bureau‘s progress in improving its cost estimation abilities. The
testimony is based on previously issued and ongoing GAO work.
What GAO Found:
The Bureau continues to make noteworthy gains in mitigating risks and
in keeping the headcount on-track, but a number of challenges remain.
Specifically, over the last few months, the Bureau has made important
strides in improving oversight of testing key IT systems. For example,
the Bureau named a testing officer to monitor the testing of census-
taking activities. The Bureau has also made progress in system testing,
but faces tight timeframes in finalizing the paper-based operations
control system (PBOCS), which will be used to manage field operations.
If any significant problems are identified during the testing phases of
PBOCS, there will be little time, in most cases, to resolve the
problems before the system needs to be deployed.
Address canvassing, an operation where temporary workers known as
listers go door–to-door to verify and update address data, finished
ahead of schedule, but was over budget. Based on initial Bureau data,
the preliminary figure on the actual cost of address canvassing is $88
million higher than the original estimate of $356 million, an overrun
of 25 percent. A key reason for the overrun is that the Bureau did not
update its cost estimates to reflect the changes to the address
canvassing workload. Further, the Bureau did not follow its staffing
strategy and hired too many listers. The Bureau‘s efforts to
fingerprint employees, which was required as part of a criminal
background check, did not proceed smoothly, in part because of training
issues. As a result, over 35,000 temporary census workers--over a fifth
of the address canvassing workforce--were hired despite the fact that
their fingerprints could not be processed and they were not fully
screened for employment eligibility. The Bureau is refining instruction
manuals and taking other steps to improve the fingerprinting process
for future operations.
GAO is unable to verify the accuracy of the $14.7 billion estimated
cost of the 2010 Census because key details and assumptions are
unavailable. However, the Bureau is taking steps to improve its cost
estimation process for 2020, including training its staff in cost
estimation skills.
While the Bureau has taken a number of actions to mitigate risk and its
overall readiness for 2010 has improved, much work remains to be done.
Many things can happen over the next few months, and keeping the entire
enterprise on-plan will continue to be a daunting challenge fraught
with risks. High levels of public participation, and continued Bureau
and congressional attention to stewardship, performance, and
accountability, will be key to a successful census.
What GAO Recommends:
GAO is not making new recommendations, but past reports recommended the
Bureau improve its cost estimation procedures, ensure the accuracy of
its address list, and conduct end-to-end testing of IT systems. The
Bureau generally agreed with the recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-10-132T] or key
components. For more information, contact Robert Goldenkoff at (202)
512-2757 or goldenkoffr@gao.gov.
[End of section]
Mr. Chairman, Senator McCain, and Members of the Subcommittee:
I am pleased to be here today to provide a progress report on the U.S.
Census Bureau's (Bureau) implementation of the 2010 Census. As you
know, in March 2008, we designated the 2010 Census a high-risk area,
citing a number of long-standing and emerging challenges including (1)
weaknesses in the Bureau's information technology (IT) acquisition and
contract management function; (2) problems with the performance of
handheld computers (HHC) that were designed in part for address
canvassing, a massive field operation where temporary census employees
go door-to-door to update the Bureau's address list of around 140
million housing units; and (3) uncertainty over the ultimate cost of
the census--now estimated at around $14.7 billion.[Footnote 1]
Overarching all of these concerns was the lack of a full dress
rehearsal, which limited the Bureau's ability to demonstrate critical
enumeration activities under near-census-like conditions, and the lack
of time to complete remaining work. By law, Census Day is April 1,
2010. As a result, the design and execution of the decennial census
proceed under a rigid schedule; there are no timeouts, no do-overs, and
no reset buttons. Collectively, these issues raised questions about the
Bureau's readiness for the 2010 Census.
This past March, exactly a year after we identified the decennial
census as a high-risk area, we appeared before this Subcommittee and
testified that the Bureau had made commendable progress in rolling out
key components of the census, making improvements to the HHCs, certain
risk management efforts, and various other activities. Nevertheless, a
number of operations and support systems still needed to be designed,
planned, and tested.[Footnote 2]
As requested, my remarks today will focus on the extent to which the
Bureau has improved its overall preparedness for the headcount, paying
particular attention to the steps the Bureau has taken since March
2009, to mitigate risks and implement critical enumeration activities.
The focus of my discussion will be (1) the rollout of key IT systems,
(2) our preliminary findings on the results of address canvassing and
the lessons learned from that operation that can be applied to
subsequent field operations, and (3) the Bureau's progress in improving
its cost estimation abilities.
Lessons learned from 2010 will also be useful for informing the design
of the next decennial census. Rigorous planning and perhaps even a
fundamental re-examination of the census might be required because the
current approach to the national enumeration may no longer be
financially sustainable. Indeed, the cost of conducting the census has,
on average, doubled each decade since 1970 in constant 2010 dollars. If
that rate of cost escalation continues into 2020, the nation could be
looking at a $30 billion census.
My testimony today is based on our ongoing and completed reviews of the
development, testing, and implementation of selected IT systems; our on-
site observations of address canvassing this past spring; and our
examination of the Bureau's efforts to improve its cost estimates.
Specifically, we analyzed key documents including plans, procedures,
and guidance for the selected activities, and interviewed cognizant
Bureau officials at headquarters and local census offices. In addition,
during address canvassing, we conducted 38 observations of address
listers and crew leaders as they went door-to-door and interviewed
local census office managers in 20 urban, suburban, and rural early
local census offices across the country. We anticipate issuing more
comprehensive reports on the results of this work in the near future.
On September 29, 2009, we provided the Bureau with a statement of facts
for ongoing audit work, and on October 1, 2009, the Bureau forwarded
written comments. The Bureau made some suggestions where additional
context or clarification was needed and, where appropriate, we made
those changes. We conducted our work in accordance with generally
accepted government auditing standards. Those standards require that we
plan and perform the audits to obtain sufficient, appropriate evidence
to provide a reasonable basis for our findings and conclusions based on
our audit objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objectives.
In summary, the Bureau continues to make noteworthy progress in
mitigating risks and keeping the decennial on track. Still, as the
Bureau is well aware, much work remains to be done, and a successful
census requires the near perfect alignment of a myriad of activities as
well as a high level of public cooperation, and even a small setback or
misstep could mushroom and potentially derail the Bureau's efforts.
More specifically, over the last few months, the Bureau has made
important strides in improving oversight of testing key IT systems,
strengthened certain risk management activities, and generally
completed address canvassing ahead of schedule. Further, we are
encouraged by the seating of a new census director this past July (this
position had been vacant for several months), as well as by the
experienced advisors he has put in place to assist him, several of whom
have experience from the 2000 Census.
That said, a number of challenges and uncertainties still need to be
addressed. For example, while the Bureau has made progress in testing
key decennial systems, critical testing activities need to be completed
before the systems will be ready to support the 2010 Census. Further,
the Bureau's ability to develop accurate and reliable cost estimates
for the census remains a concern. For example, based on initial Bureau
data, the preliminary figure on the actual cost of address canvassing
is $88 million (25 percent) higher than the original estimate of $356
million. Moreover, the Bureau's efforts to fingerprint employees, which
was required as part of a criminal background check, did not proceed
smoothly, and over 35,000 temporary census workers--over a fifth of the
address canvassing workforce--were hired despite the fact that their
fingerprints could not be processed, in part because many were
illegible.
Overall, while there have been many positive developments in the last
few months, the 2010 Census remains a high risk area because of the
amount of work that still needs to be completed under a very tight time
frame, as well as for the inherent uncertainties in managing such a
complex enterprise, including the ultimate level of public
participation. Public engagement along with continued congressional and
Bureau attention to stewardship, performance, and accountability are
key to a successful census.
Background:
As you know, Mr. Chairman, the decennial census is a constitutionally
mandated enterprise critical to our nation. Census data are used to
apportion seats and redraw Congressional districts, and to help
allocate over $400 billion in federal aid to state and local
governments each year.
We added the 2010 Census to our list of high-risk areas in March 2008
because improvements were needed in the Bureau's management of IT
systems, the reliability of the HHCs, and the quality of the Bureau's
cost estimates. Compounding the risk was that the Bureau canceled a
full dress rehearsal of the census that was scheduled in 2008, in part,
because of the HHC's performance problem, which included freeze-ups and
unreliable data transmissions. Although the Bureau had planned to use
the HHCs to collect data for both address canvassing and in going door
to door following up with nonrespondents, the Bureau ultimately decided
to use the HHCs for address canvassing and revert to collecting
nonresponse follow-up data using paper.[Footnote 3] As a result of this
decision, the Bureau had to redesign components of its field data
collection system to accommodate the new approach, thus introducing new
risks.
Among other actions, in response to our findings and recommendations,
the Bureau strengthened its risk management efforts, including the
development of a high-risk improvement plan that described the Bureau's
strategy for managing risk and key actions to address our concerns.
Still, in March 2009, in testimony before this Subcommittee, we
continued to question the Bureau's readiness. Specifically, we noted
that with little more than a year remaining until Census Day,
uncertainties surrounded critical operations and support systems, and
the Bureau lacked sufficient policies, procedures, and trained staff to
develop high-quality cost estimates. Moving forward, we said that it
will be essential for the Bureau to develop plans for testing systems
and procedures not included in the dress rehearsal, and for Congress to
monitor the Bureau's progress.
The Bureau Has Made Progress on the Management and Testing of Key IT
Systems, but Little Time Remains to Address Outstanding Issues:
Since 2005, we have reported on weaknesses in the Bureau's management
of its IT acquisitions, and issues continue concerning the Bureau's IT
management and testing of key 2010 Census systems. In March 2009, we
reported and testified that while the Bureau took initial steps to
enhance its program-wide oversight of testing activities, those steps
had not been sufficient.[Footnote 4] Furthermore, while the Bureau had
made progress in testing key decennial systems, critical testing
activities remained to be performed before they would be ready to
support the 2010 Census. At that time we recommended that the Bureau
improve its oversight of the completion of testing activities for key
systems.
In response to our findings and recommendations, the Bureau has taken
several steps to improve its management of IT for the 2010 Census. For
example, the Bureau named a Decennial Census Testing Officer whose
primary responsibilities include monitoring testing for decennial
census activities. In order to help improve the rigor and quality of
test planning and documentation, this official leads a bimonthly
process to consolidate and evaluate test planning and status across all
key decennial census operations, resulting in a decennial census
testing overview document.
With respect to system testing, progress is being made, but much
testing remains to be completed as shown in the following table.
Table 1: Status of key testing activities:
Census system: Headquarters Processing - Universe Control and
Management;
Status of testing activities: System development is divided into three
phases. According to the Bureau, testing for the first of three phases
has completed and the second phase is being tested.
Census system: Headquarters Processing - Response Processing System;
Status of testing activities: Testing is not anticipated to start until
November 2009.
Census system: Master Address File/Topologically Integrated Geographic
Encoding and Referencing (MAF/TIGER) System;
Status of testing activities: Five of eight test plans for 2010
Operations have been baselined. Testing activities for one baselined
test plan (address canvassing) have been completed; three are under
way; and one has not yet started. The approach to test metrics for
MAF/TIGER has recently been revised; however, only two of five
baselined test plans include detailed metrics.
Census system: Field Data Collection Automation (FDCA);
Status of testing activities: Address canvassing operation completed,
and map printing software deployed to field offices. The FDCA
contractor is supporting the group-quarters validation operation[A] and
map printing activities.
Census system: Decennial Response Integration System;
Status of testing activities: Four of five increments have been tested.
A sixth increment was added to account for late changes. Two of four
rounds of additional operational testing are under way. The program has
completed testing for 7 of 16 interfaces, but has experienced delays in
testing the remaining interfaces.
Census system: Paper-Based Operations Control System (PBOCS);
Status of testing activities: System development has been divided into
three major releases, following a preliminary release in preparation
for a limited end-to-end test in June 2009. Defects identified during
this test are being reworked. Testing of the first major release, as
well as an additional limited end-to-end test, are under way. In
addition, the Bureau is planning and developing software for the
remaining releases. Program officials state the limited time to
complete development and testing remains a challenge.
Census system: Data Access and Dissemination System II;
Status of testing activities: The system consists of two subsystems,
each with three iterations of development and testing. For one
subsystem, the program is testing the second of the three iterations.
For the other subsystem, the program is developing the second of three
iterations and plans to begin testing this iteration in early 2010.
Development and testing is proceeding according to schedule.
Source: GAO analysis of U.S. Census Bureau data.
[A] Group-quarters validation entails validating addresses identified
as potential group quarters, such as college residence halls and group
homes.
[End of table]
The Bureau has also made progress in end-to-end testing, but
substantial work remains to be completed. For example, the Bureau has
completed limited end-to-end tests for nonresponse follow-up and group-
quarters enumeration[Footnote 5] on the Paper-Based Operations Control
System[Footnote 6] (PBOCS), a work flow management system the Bureau
developed late in the census cycle when it moved from the HHCs to a
paper-based approach to nonresponse follow-up and other field
operations. However, Bureau officials stated that, although they were
satisfied with the results of the tests, significant additional testing
will be needed. For example, several critical issues were identified
during these tests that will need to be resolved and retested. In
addition, the test was not designed to evaluate the level of system
performance needed while processing the estimated 48 million housing
units that will be in the nonresponse-follow-up workload. According to
the Bureau, a performance test is being designed for the first major
release; however, detailed plans for this test have not yet been
completed. Finally, the test was performed with experienced census
employees, while the system will be used by newer, temporary employees.
[Footnote 7]
Given the importance of IT systems to the decennial census, it is
critical that the Bureau ensure these systems are thoroughly tested.
Bureau officials have repeatedly stated that the limited amount of time
remaining will make completing all testing activities challenging.
The Bureau Needs to Prioritize Remaining Needs for Systems Used to
Manage Field Data Collection:
The Bureau faces significant challenges finalizing PBOCS. Most notably,
the Bureau needs to determine the remaining detailed requirements for
the system to be developed. As of early September 2009, the Bureau had
established high-level requirements for its PBOCS but had not yet
finalized the detailed requirements. High-level requirements describe
in general terms what functions the system will accomplish, such as
producing specific management reports on the progress of specific paper-
based operations or checking-out and checking-in groups of census forms
for shipping or processing. Detailed requirements describe more
specifically what needs to be done in order to accomplish such
functions. For PBOCS, such detailed requirements might include, for
example, which data from which data source should be printed where on a
specific management report. According to Bureau officials, in the
absence of such specificity in the requirements for the 2008 dress
rehearsal, contract programmers with little decennial census experience
made erroneous assumptions about which data to use when preparing some
quality control reports. As a result, quality assurance managers were
unable to rely on the reports for tracking progress.
In recognition of the serious implications that shortcomings in PBOCS
would have for the conduct of the 2010 Census and to see whether there
were additional steps that could be taken to mitigate the outstanding
risks to successful PBOCS development and testing, in June 2009, the
Bureau chartered an assessment of PBOCS, chaired by the Bureau's chief
information officer (CIO). The assessment team reported initially in
late July 2009 and provided an update the following month. The review
stated that the PBOCS developers had made a strong effort to involve
the system stakeholders in the development process. However, the review
also identified several concerns with PBOCS development. For example,
the review found and we confirmed that the Bureau could improve its
requirements management for PBOCS. According to the CIO, the Bureau has
taken steps to address some of these findings, such as providing
additional resources for testing and development; however, resolving
problems found during testing before the systems need to be deployed
will be a challenge.
At the end of our review, the Bureau presented evidence of the steps it
had taken to document and prioritize requirements. We did not assess
the effectiveness of these steps. Until the Bureau completes the
detailed requirements for PBOCS, it will not have reasonable assurance
that PBOCS will meet the program's needs. The Bureau is continuing to
examine how improvements will be made.
The Bureau Generally Completed Address Canvassing Ahead of Schedule But
Went Over Budget:
A successful census relies on an accurate list of all addresses where
people live in the country, because it identifies all households that
are to receive a census questionnaire and serves as a control mechanism
for following up with households that fail to respond. If the address
list is inaccurate, people can be missed, counted more than once, or
included in the wrong location.
Address canvassing is one of several procedures the Bureau uses to help
ensure an accurate address list and, because it is based on on-site
verification, it is particularly important for identifying the
locations of nontraditional or "hidden" housing units such as converted
attics and basements. Although these types of dwellings have always
existed, the large number of foreclosures the nation has recently
experienced, as well as the natural disasters that have hit the Gulf
Coast and other regions, have likely increased the number of people
doubling-up, living in motels, cars, tent cities, and other less
conventional living arrangements. Such individuals are at greater risk
of being missed in the census.
The Bureau conducted address canvassing from March to July 2009. During
that time, about 135,000 address listers went door to door across the
country, comparing the housing units they saw on the ground to what was
listed in the database of their HHCs. Depending on what they observed,
listers could add, delete, or update the location of housing units.
Although the projected length of the field operation ranged from nine
to fourteen weeks, most early local census offices completed the effort
in less than 10 weeks. Moreover, the few areas that did not finish
early were delayed by unusual circumstances such as access issues
created by flooding. The completion rate is a remarkable accomplishment
given the HHC's troubled history. The testing and improvements the
Bureau made to the reliability of the HHCs prior to the start of
address canvassing, including a final field test that was added to the
Bureau's preparations in December 2008, played a key role in the pace
of the operation, but other factors, once address canvassing was
launched, were important as well, including the (1) prompt resolution
of problems with the HHCs as they occurred and (2) lower than expected
employee turnover.
With respect to the prompt resolution of problems, although the
December 2008 field test indicated that the more significant problems
affecting the HHCs had been resolved, various glitches continued to
affect the HHCs in the first month of the operation. For example, we
were informed by listers or crew leaders in 14 early local census
offices that they had encountered problems with transmissions, freeze-
ups, and other problems. Moreover, in 10 early local census offices we
visited, listers said they had problems using the Global Positioning
System function on their HHCs to precisely locate housing units. When
such problems occurred, listers called their crew leaders and the
Bureau's help desk troubleshooted the problems. When the issues were
more systemic in nature, such as a software issue, the Bureau was able
to quickly fix them using software patches.
Moreover, to obtain an early warning of trouble, the Bureau monitored
key indicators of the performance of the HHCs such as the number of
successful and failed HHC transmissions. This approach proved useful as
Bureau quality control staff were alerted to the existence of a
software problem when they noticed that the devices were taking a long
time to close out completed assignment areas.
The Bureau also took steps to address procedural issues. For example,
in the course of our field observations, we noticed that in several
locations listers were not always adhering to training for identifying
hidden housing units. Specifically, listers were instructed to knock on
every door and ask, "Are there any additional places in this building
where people live or could live?" However, we found that listers did
not always ask this question. On April 28, 2009, we discussed this
issue with senior Bureau officials. The Bureau, in turn, transmitted a
message to listers' HHCs emphasizing the importance of following
training and querying residents if possible.
Lower than expected attrition rates and listers' availability to work
more hours than expected also contributed to the Bureau's ability to
complete the address canvassing operation ahead of schedule. For
example, the Bureau had planned for 25 percent of new hires to quit
before, during, or soon after training; however, the national average
was 16 percent. Bureau officials said that not having to replace
listers with inexperienced staff accelerated the pace of the operation.
Additionally, the Bureau assumed that employees would be available 18.5
hours a week. Instead, they averaged 22.3 hours a week.
The Bureau's address list at the start of address canvassing consisted
of 141.8 million housing units. Listers added around 17 million
addresses and marked about 21 million for deletion because, for
example, the address did not have a structure. All told, listers
identified about 4.5 million duplicate addresses, 1.2 million
nonresidential addresses, and about 690,000 addresses that were
uninhabitable structures. Importantly, these preliminary results
represent actions taken during the production phase of address
canvassing and do not reflect actual changes made to the Bureau's
master address list as the actions are first subject to a quality
control check and then processed by the Bureau's Geography Division.
The preliminary analysis of addresses flagged for add and delete shows
that the results of the operation (prior to quality control) were
generally consistent with the results of address canvassing for the
2008 dress rehearsal. Table 2 compares the add and delete actions for
the two operations.
Table 2: Percentage of Add and Delete Lister Actions (Prior to Quality
Control or Bureau Processing) for 2010 Address Canvassing and 2008
Dress Rehearsal Address Canvassing:
Adds:
2010 Address Canvassing: 10.8%;
2008 Dress Rehearsal Address Canvassing: 7.3%.
Deletes:
2010 Address Canvassing: 13.2%;
2008 Dress Rehearsal Address Canvassing: 15.1%.
Source: GAO analysis of U.S. Census Bureau data.
[End of table]
Address Canvassing Costs Exceeded Budget Because of Unanticipated
Workload and Hiring:
According to the Bureau's preliminary analysis, the estimated cost for
address canvassing field operations was $444 million, or $88 million
(25 percent) more than its initial budget of $356 million.[Footnote 8]
As shown in table 3, according to the Bureau, the cost overruns were
because of several factors.
Table 3: Bureau's Preliminary Analysis of Address Canvassing Costs
Exceeding Budget:
Reasons for exceeding budget: Increased Initial Workload;
Estimated costs (in millions): $41.
Reasons for exceeding budget: Underestimated Quality Control Workload;
Estimated costs (in millions): $34.
Reasons for exceeding budget: Training Additional Staff;
Estimated costs (in millions): $7.
Reasons for exceeding budget: Fingerprinting (funded separately);
Estimated costs (in millions): $6.
Reasons for exceeding budget: Total;
Estimated costs (in millions): $88.
Source: U.S. Census Bureau.
[End of table]
One such factor was that the address canvassing cost estimate was not
comprehensive, which resulted in a cost increase of $41 million. The
Bureau inadvertently excluded 11 million addresses identified in
address file updates from the initial address canvassing workload and
fiscal year 2009 budget. Further, the additional 11 million addresses
increased the Bureau's quality control workload, where the Bureau
verifies certain actions taken to correct the address list.
Specifically, the Bureau failed to anticipate the impact these
addresses would have on the quality control workload and therefore did
not revise its cost estimate accordingly. Moreover, under the Bureau's
procedures, addresses that failed quality control would need to be
recanvassed, but the Bureau's cost model did not account for the extra
cost of recanvassing of any addresses. As a result, the Bureau
underestimated its quality control workload by 26 million addresses
which resulted in $34 million in additional costs, according to the
Bureau.
Bringing aboard more staff than was needed also contributed to the cost
overruns. For example, according to the Bureau's preliminary analysis,
training additional staff accounted for about $7 million in additional
costs.[Footnote 9] Bureau officials attributed the additional training
cost to inviting additional candidates to initial training because of
concerns that recruiting and hiring staff would be problematic, even
though (1) the Bureau's staffing goals already accounted for the
possibility of high turnover and (2) the additional employees were not
included in the cost estimate or budget.
The largest field operation will be nonresponse follow-up, when the
Bureau is to go door to door in an effort to collect data from
households that did not mail back their census questionnaire. Over
570,000 enumerators will need to be hired for that operation. To better
manage the risk of staffing difficulties while simultaneously
controlling costs, several potential lessons learned can be drawn from
the Bureau's experience during address canvassing. For example, we
found that the staffing authorization and guidance provided to some
local census managers were unclear and did not specify that there was
already a cushion in the hiring goals for local census offices to
account for potential turnover. Also, basing the number of people
invited to initial training on factors likely to affect worker hiring
and retention, such as the local employment rate, could help the Bureau
better manage costs.
According to Bureau officials, they are reviewing the results from
address canvassing to determine whether they need to revisit the
staffing strategy for nonresponse follow-up and have already made some
changes. For example, in recruiting candidates, when a local census
office reaches 90 percent of its qualified applicant goal, it is to
stop blanket recruiting and instead focus its efforts on areas that
need more help, such as tribal lands. However, in hiring candidates,
the officials pointed out that they are cautious not to underestimate
resource needs for nonresponse follow-up based on address canvassing
results because they face different operational challenges in that
operation than for address canvassing. For example, for nonresponse
follow-up, the Bureau needs to hire enumerators who can work in the
evenings when people are more likely to be at home and who can
effectively deal with reluctant respondents, whereas with address
canvassing, there was less interaction with households and the
operation could be completed during the day.
Address Canvassing Cost Overruns Are Symptomatic of Weaknesses with
Cost Estimation Efforts:
Problems with accurately estimating the cost of address canvassing are
indicative of long-standing weaknesses in the Bureau's ability to
develop credible and accurate cost estimates for the 2010 Census.
Accurate cost estimates are essential to a successful census because
they help ensure that the Bureau has adequate funds and that Congress,
the administration, and the Bureau itself can have reliable information
on which to base decisions. However, in our past work, we noted that
the Bureau's estimate lacked detailed documentation on data sources and
significant assumptions, and was not comprehensive because it did not
include all costs.[Footnote 10] Following best practices from our Cost
Estimating and Assessment Guide, such as defining necessary resources
and tasks, could have helped the Bureau recognize the need to update
address canvassing workload and other operational assumptions,
resulting in a more reliable cost estimate.[Footnote 11]
Given the Bureau's past difficulties in developing credible and
accurate cost estimates, we are concerned about the reliability of the
figures that were used to support the 2010 budget, especially the costs
of nonresponse follow-up, which is estimated to cost $2.7 billion. We
have discussed the cost estimate for nonresponse follow-up with Bureau
officials, and they have said they are looking to see how foreclosures
and vacant housing units might affect the nonresponse follow-up
workload. In addition, Bureau officials said they will analyze address
canvassing data and determine if there are any implications for future
operations.
Nevertheless, there still remains a great deal of uncertainty around
the final cost of the 2010 Census. In part, this is because of changes
made to the census design after April 2008, when the Bureau reverted to
a paper-based data collection method for nonresponse follow-up in
response to the performance problems with the HHCs. The uncertainty
also stems from the fact that the assumptions used to develop the
revised cost estimate were not tested during the 2008 dress rehearsal.
According to budget documents, after the decision to return to a paper-
based nonresponse follow-up, the life cycle cost estimate increased by
over $3 billion dollars.
Moving forward, it will be important for the Bureau to ensure the
reliability of the 2020 cost estimate, and the Bureau has already taken
several actions in that regard. For example, based on recommendations
from our June 2008 report, the Bureau plans to train its staff on cost
estimation skills, including conducting uncertainty analysis. In
addition, the Bureau is developing the Decennial Budget Integration
Tool (DBiT), which according to the Bureau, should consolidate budget
information and enable the Bureau to better document its cost
estimates. Officials said that DBiT is capturing actual fiscal year
2009 costs, which will be used to estimate the life cycle cost for the
2020 census. However, officials also said that DBiT needs further
testing, and may not be fully used until the 2012 budget.
Bureau Needs to Improve Its Policies and Procedures for Fingerprinting
Temporary Employees:
To better screen its workforce of hundreds of thousands of temporary
census workers, the Bureau plans to fingerprint its temporary workforce
for the first time in the 2010 Census.[Footnote 12] In past censuses,
temporary workers were only subject to a name background check that was
completed at the time of recruitment. The Federal Bureau of
Investigation (FBI) is to provide the results of a name background
check when temporary workers are first recruited. At the end of the
workers' first day of training, Bureau employees who have received
around 2 hours of fingerprinting instruction are to capture two sets of
ink fingerprint cards. The cards are then sent to the Bureau's National
Processing Center in Jeffersonville, Indiana, to be scanned and
electronically submitted to the FBI. If the results show a criminal
record that makes an employee unsuitable for employment, the Bureau is
to either terminate the person immediately or place the individual in
nonworking status until the matter is resolved. If the first set of
prints are unclassifiable, the National Processing Center is to send
the FBI the second set of prints.
However, fingerprinting during address canvassing was problematic. Of
the over 162,000 employees hired for the operation, 22 percent--or
approximately 35,700 workers--had unclassifiable prints that the FBI
could not process. The FBI determined that the unclassifiable prints
were generally the result of errors that occurred when the prints were
first made. Factors affecting the quality of the prints included
difficulty in first learning how to effectively capture the prints and
the adequacy of the Bureau's training. Further, the workspace and
environment for taking fingerprints was unpredictable, and factors such
as the height of the workspace on which the prints were taken could
affect the legibility of the prints.
Consistent with FBI guidance, the Bureau relied solely on the results
of the name background check for the nearly 36,000 employees with
unclassifiable prints.[Footnote 13] However, it is possible that more
than 200 people with unclassifiable prints had disqualifying criminal
records but still worked, and had contact with the public during
address canvassing.[Footnote 14] Indeed, of the prints that could be
processed, fingerprint results identified approximately 1,800 temporary
workers (1.1 percent of total hires) with criminal records that name
check alone failed to identify. Of the 1,800 workers with criminal
records, approximately 750 (42 percent) were terminated or were further
reviewed because the Bureau determined their criminal records--which
included crimes such as rape, manslaughter, and child abuse--
disqualified them from census employment.
Projecting these percentages to the 35,700 temporary employees with
unclassifiable prints, it is possible that more than 200 temporary
census employees might have had criminal records that would have made
them ineligible for census employment. Applying these same percentages
to the approximately 600,000 people the Bureau plans to fingerprint for
nonresponse follow-up, unless the problems with fingerprinting are
addressed, we estimate that approximately 785 employees with
unclassifiable prints could have disqualifying criminal records but
still end up working for the Bureau.[Footnote 15]
Aside from public safety concerns, there are cost issues as well. The
FBI charged the Bureau $17.25 per person for each background check,
whether or not the fingerprints were classifiable.
The Bureau stated that it has taken steps to improve image quality for
fingerprints captured in future operations by refining instruction
manuals and providing remediation training on proper procedures. In
addition, the Bureau is considering activating a feature on the
National Processing Center's scanners that can check the legibility of
the image and thus prevent poor quality prints from reaching the FBI.
These are steps in the right direction. As a further contingency, it
might also be important for the Bureau to develop a policy for re-
fingerprinting employees to the extent that both cards cannot be read.
The Bureau Used Enhanced Training and Guidance for Canvassing Hurricane
Affected Areas:
The scale of the destruction in those areas affected Hurricanes
Katrina, Rita, and Ike made address canvassing in parts of Mississippi,
Louisiana, and Texas, especially challenging (see fig. 1). Hurricane
Katrina alone destroyed or made uninhabitable an estimated 300,000
homes. Recognizing the difficulties associated with address canvassing
in these areas because of shifting and hidden populations and changes
to the housing stock, the Bureau, partly in response to recommendations
made in our June 2007 report,[Footnote 16] developed supplemental
training materials for natural disaster areas to help listers identify
addresses where people are, or may be, living when census
questionnaires are distributed. For example, the materials noted the
various situations listers might encounter, such as people living in
trailers, homes marked for demolition, converted buses and recreational
vehicles, and nonresidential space such as storage areas above
restaurants. The training material also described the clues that could
alert listers to the presence of non-traditional places where people
are living and provided a script they should follow when interviewing
residents on the possible presence of hidden housing units.
Figure 1: Locating and Counting People Displaced By Storms Presents A
Challenge:
[Refer to PDF for image: two photographs of dwellings]
Source: GAO.
[End of figure]
Additional steps taken by the city of New Orleans also helped the
Bureau overcome the challenge of canvassing neighborhoods devastated by
Hurricane Katrina. As depicted in fig. 2 below, city officials replaced
the street signs even in abandoned neighborhoods. This assisted listers
in locating the blocks they were assigned to canvass and expedited the
canvassing process in these deserted blocks.
Figure 2: Replacement Street Signs Facilitated Address Canvassing in
New Orleans:
[See PDF for image: photograph]
Source: GAO.
[End of figure]
To further ensure a quality count in the hurricane affected areas, the
Bureau plans to hand-deliver an estimated 1.2 million questionnaires
(and simultaneously update the address list) to housing units in much
of southeast Louisiana and south Mississippi that appear inhabitable,
even if they do not appear on the address list updated by listers
during address canvassing. Finally, the Bureau stated that it must
count people where they are living on Census Day and emphasized that if
a housing unit gets rebuilt and people move back, then that is where
those people will be counted. However, if they are living someplace
else, then they will be counted where they are living on Census Day.
Concluding Observations:
The Bureau has made remarkable progress in improving its overall
readiness for 2010, with substantial strides being made in the
management of its IT systems and other areas. That said, as I noted
throughout this statement, considerable challenges and uncertainties
lie ahead. While the decennial is clearly back on track, many things
can happen over the next few months, and keeping the entire enterprise
on plan continues to be a daunting challenge fraught with risks.
Mr. Chairman and members of this Subcommittee, this concludes my
statement. I would be happy to respond to any questions that you might
have at this time.
GAO Contacts:
If you have any questions on matters discussed in this statement,
please contact Robert N. Goldenkoff at (202) 512-2757 or by e-mail at
goldenkoffr@gao.gov. Other key contributors to this testimony include
Steven Berke, Virginia Chanley, Benjamin Crawford, Jeffrey DeMarco,
Dewi Djunaidy, Vijay D'Souza, Elizabeth Fan, Ronald Fecso, Amy Higgins,
Richard Hung, Kirsten Lauber, Jason Lee, Andrea Levine, Signora May, Ty
Mitchell, Naomi Mosser, Catherine Myrick, Lisa Pearson, David Powner,
David Reed, Jessica Thomsen, Jonathan Ticehurst, Shaunyce Wallace,
Timothy Wexler, and Katherine Wulff.
[End of section]
Footnotes:
[1] GAO, Information Technology: Significant Problems of Critical
Automation Program Contribute to Risks Facing 2010 Census, [hyperlink,
http://www.gao.gov/products/GAO-08-550T] (Washington, D.C.: Mar. 5,
2008).
[2] GAO, 2010 Census: Fundamental Building Blocks of a Successful
Enumeration Face Challenges, [hyperlink,
http://www.gao.gov/products/GAO-09-430T] (Washington, D.C.: Mar. 5,
2009).
[3] Nonresponse follow-up is the largest field operation which entails
enumerators following up with nonrespondents through personal
interviews to complete paper questionnaires.
[4] GAO, Information Technology: Census Bureau Testing of 2010
Decennial Systems Can Be Strengthened, [hyperlink,
http://www.gao.gov/products/GAO-09-262] (Washington, D.C.: Mar. 5,
2009) and GAO, Information Technology: Census Bureau Needs to
Strengthen Testing of 2010 Decennial Systems, [hyperlink,
http://www.gao.gov/products/GAO-09-413T] (Washington, D.C.: Mar. 5,
2009).
[5] Group-quarters enumeration entails enumerators collecting
information from people living in places such as college residence
halls, prisons and group homes.
[6] The PBOCS includes IT systems and infrastructure needed to support
the use of paper forms for operations such as group quarters
enumeration and nonresponse follow-up.
[7] According to the Bureau, an additional limited end-to-end test that
is under way is utilizing clerical staff from local census offices.
[8] Address canvassing costs for field operations include training,
work hours, and mileage for temporary field staff. These costs do not
include recruiting, large block canvassing, office infrastructure,
management or technical support staff, IT contracts, and partnership
program or communication campaign activities.
[9] Officials clarified that training costs should exclude training
hours spent for fingerprinting and conducting 4 hours of actual
production work as part of training.
[10] GAO, 2010 Census: Census Bureau Should Take Action to Improve the
Credibility and Accuracy of Its Cost Estimate for the Decennial Census,
[hyperlink, http://www.gao.gov/products/GAO-08-554] (Washington, D.C.:
June 16, 2008).
[11] GAO, GAO Cost Estimating And Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009).
[12] The National Crime Prevention and Privacy Compact, enacted in
1998, generally requires that fingerprints be submitted with all
requests for criminal history record checks for noncriminal justice
purposes; 42 U.S.C. § 14616. For the 2000 Census, the FBI did not have
the capacity to timely process the fingerprints of Census‘s temporary
workforce, so they were subject to only a name background check.
[13] The Bureau will refingerprint employees with unclassifiable prints
if they are rehired for another operation.
[14] The Bureau uses its adjudication criteria to determine if
applicants' criminal history background present an unacceptable risk to
the Census.
[15] The approximately 600,000 workers to be fingerprinted for
nonresponse follow-up include over 570,000 enumerators and other field
staff, such as crew leaders and field operation supervisors.
[16] GAO, 2010 Census: Census Bureau Has Improved the Local Update of
Census Addresses Program, but Challenges Remain, [hyperlink,
http://www.gao.gov/products/GAO-07-736] (Washington, D.C.: June 14,
2007).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: