2010 Census
Fundamental Building Blocks of a Successful Enumeration Face Challenges
Gao ID: GAO-09-430T March 5, 2009
The decennial census is a constitutionally-mandated activity that produces data used to apportion congressional seats, redraw congressional districts, and allocate billions of dollars in federal assistance. In March 2008, GAO designated the 2010 Census a high-risk area in part because of problems with the performance of handheld computers used to collect data. The U.S. Census Bureau has since strengthened its risk management efforts and made other improvements; however, the Bureau curtailed a dress rehearsal scheduled for 2008 and was unable to test key operations under census-like conditions. This testimony discusses the Bureau's readiness for 2010 and covers: (1) importance of reliable cost estimates; (2) building a complete and accurate address list; (3) following up on missing and conflicting responses to ensure accuracy; (4) targeting outreach to undercounted populations; and (5) designing, testing, and implementing technology for the census. The testimony is based on previously issued and ongoing GAO work.
The decennial census is an inherently fragile undertaking, requiring many moving parts to come together in a short time frame. For example, accurate cost estimates help ensure that the Bureau has adequate funds, and that Congress, the administration, and the Bureau itself have reliable information on which to base advice and decisions. However, as GAO has reported before, the Bureau has insufficient policies and procedures and inadequately trained staff for conducting high-quality cost estimation for the decennial census. A successful census requires a complete and accurate address list. The Bureau sends thousands of census workers (listers) into the field to collect and verify address information, and this year for the first time, listers will use handheld computers to collect data. During the dress rehearsal there were significant technical problems. A small-scale field test showed that these problems appear to have been addressed; however, the test was not carried out under full census-like conditions and did not validate all address canvassing requirements. Nonresponse follow-up, the Bureau's largest and most costly field operation, was initially planned to be conducted using the handheld computers, but was recently changed to a paper-based system due to technology issues. The Bureau has not yet developed a road map for monitoring the development and implementation of nonresponse follow-up under the new design. Such a plan is essential to conducting a successful nonresponse follow-up. Furthermore, the system that manages the flow of work in field offices is not yet developed. Lacking plans for the development of both nonresponse follow-up and this management system, the Bureau faces the risk of not having them developed and fully tested in time for the 2010 Census. In an effort to reduce the undercount, the Bureau is implementing a program of paid advertising integrated with other communications strategies, such as partnerships with state, local, and tribal governments and community organizations. Moving toward 2010, the Bureau faces long-standing challenges with the nation's linguistic diversity and privacy concerns, which can contribute to the undercounting of some groups. Since 2005, GAO has reported concerns with the Bureau's management and testing of key IT systems. GAO is reviewing the status and plans for the testing of key 2010 Census systems, and while the Bureau has made progress in conducting systems, integration, and end-to-end testing, critical testing still remains to be performed before systems will be ready to support the 2010 Census, and the planning for the testing needs much improvement. In short, while the Bureau has made some noteworthy progress in gearing up for the enumeration, with just over a year remaining until census day, uncertainties surround the Bureau's overall readiness for 2010.
GAO-09-430T, 2010 Census: Fundamental Building Blocks of a Successful Enumeration Face Challenges
This is the accessible text file for GAO report number GAO-09-430T
entitled '2010 Census: Fundamental Building Blocks of a Successful
Enumeration Face Challenges' which was released on March 6, 2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Federal Financial Management, Government
Information, Federal Services and International Security, Committee on
Homeland Security and Governmental Affairs, U.S. Senate:
United States Government Accountability Office:
GAO:
For Release on Delivery:
Expected at 2:30 p.m. EST:
Thursday, March 5, 2009:
2010 Census:
Fundamental Building Blocks of a Successful Enumeration Face
Challenges:
Statement of Robert Goldenkoff:
Director, Strategic Issues:
David A. Powner:
Director, Information Technology Management Issues:
GAO-09-430T:
GAO Highlights:
Highlights of GAO-09-430T, a testimony before the Subcommittee on
Federal Financial Management, Government Information, Federal Services
and International Security, Committee on Homeland Security and
Governmental Affairs, U.S. Senate.
Why GAO Did This Study:
The decennial census is a constitutionally-mandated activity that
produces data used to apportion congressional seats, redraw
congressional districts, and allocate billions of dollars in federal
assistance. In March 2008, GAO designated the 2010 Census a high-risk
area in part because of problems with the performance of handheld
computers used to collect data. The U.S. Census Bureau has since
strengthened its risk management efforts and made other improvements;
however, the Bureau curtailed a dress rehearsal scheduled for 2008 and
was unable to test key operations under census-like conditions. This
testimony discusses the Bureau‘s readiness for 2010 and covers: (1)
importance of reliable cost estimates; (2) building a complete and
accurate address list; (3) following up on missing and conflicting
responses to ensure accuracy; (4) targeting outreach to undercounted
populations; and (5) designing, testing, and implementing technology
for the census. The testimony is based on previously issued and ongoing
GAO work.
What GAO Found:
The decennial census is an inherently fragile undertaking, requiring
many moving parts to come together in a short time frame. For example,
accurate cost estimates help ensure that the Bureau has adequate funds,
and that Congress, the administration, and the Bureau itself have
reliable information on which to base advice and decisions. However, as
GAO has reported before, the Bureau has insufficient policies and
procedures and inadequately trained staff for conducting high-quality
cost estimation for the decennial census.
A successful census requires a complete and accurate address list. The
Bureau sends thousands of census workers (listers) into the field to
collect and verify address information, and this year for the first
time, listers will use handheld computers to collect data. During the
dress rehearsal there were significant technical problems. A small-
scale field test showed that these problems appear to have been
addressed; however, the test was not carried out under full census-like
conditions and did not validate all address canvassing requirements.
Nonresponse follow-up, the Bureau‘s largest and most costly field
operation, was initially planned to be conducted using the handheld
computers, but was recently changed to a paper-based system due to
technology issues. The Bureau has not yet developed a road map for
monitoring the development and implementation of nonresponse follow-up
under the new design. Such a plan is essential to conducting a
successful nonresponse follow-up. Furthermore, the system that manages
the flow of work in field offices is not yet developed. Lacking plans
for the development of both nonresponse follow-up and this management
system, the Bureau faces the risk of not having them developed and
fully tested in time for the 2010 Census.
In an effort to reduce the undercount, the Bureau is implementing a
program of paid advertising integrated with other communications
strategies, such as partnerships with state, local, and tribal
governments and community organizations. Moving toward 2010, the Bureau
faces long-standing challenges with the nation‘s linguistic diversity
and privacy concerns, which can contribute to the undercounting of some
groups.
Since 2005, GAO has reported concerns with the Bureau‘s management and
testing of key IT systems. GAO is reviewing the status and plans for
the testing of key 2010 Census systems, and while the Bureau has made
progress in conducting systems, integration, and end-to-end testing,
critical testing still remains to be performed before systems will be
ready to support the 2010 Census, and the planning for the testing
needs much improvement. In short, while the Bureau has made some
noteworthy progress in gearing up for the enumeration, with just over a
year remaining until census day, uncertainties surround the Bureau‘s
overall readiness for 2010.
What GAO Recommends:
GAO is not making new recommendations, but past reports recommended the
Bureau improve its cost estimation procedures and accuracy of its
address list, take steps to ensure the readiness of handheld computers,
better manage its partnership programs, and conduct end-to-end testing
of IT systems. The Bureau generally agreed with the recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-09-430T] or key
components. For more information, contact Robert Goldenkoff at (202)
512-2757 or goldenkoffr@gao.gov or David Powner at (202) 512-9286 or
pownerd@gao.gov.
[End of section]
Mr. Chairman, Senator McCain, and Members of the Subcommittee:
We are pleased to be here today to discuss with you the progress the
U.S. Census Bureau (Bureau) has made in implementing the 2010 Census,
and some of the critical challenges that lie ahead. We have been
reviewing the national enumeration for decades on behalf of Congress.
Over the years, through scores of reports and testimonies, we have
acquired broad institutional knowledge that gives us a historical view
of lessons learned for implementing a successful census. As requested,
our remarks today will focus on the current challenges the Bureau faces
and how lessons learned from prior decennials can help produce a more
cost-effective headcount.
Perhaps the most important lesson learned from past enumerations is
that the census is large, complex, and its many procedures are
interrelated, thus making it inherently fragile. An accurate population
count requires the alignment of a myriad of factors that include the
successful execution of dozens of census-taking operations, the
public's willingness to cooperate with enumerators, and the Bureau's
ability to effectively partner with thousands of state, local, and
tribal governments, as well as community and other organizations. The
bottom line is that while the census is under way, the tolerance for
any breakdowns is quite small. In light of this difficult operational
environment, effective stewardship of the Bureau is essential to help
ensure the census stays on track and the agency continues to embrace a
culture of performance and accountability. Key to this will be the
timely appointment of a Census Director who is an efficient
administrator, a respected technical professional, a strategic leader,
and capable of working constructively with Congress, officials at all
levels of government, as well as nongovernmental organizations and the
statistical community.
Other key valuable lessons learned include the importance of (1) sound
risk management, (2) staying on schedule, (3) and conducting the census
as a shared national undertaking involving Congress, government
agencies at all levels, and the public at large. One or more of these
lessons learned can be applied to the challenges the Bureau currently
faces as it gears-up for the 2010 Census.
As you know, the census has encountered several significant operational
challenges to date. Today is the first anniversary of when we first put
the 2010 Census on our high-risk list because of (1) long-standing
weaknesses in the Bureau's information technology (IT) acquisition and
contract management function, (2) problems with the performance of
handheld computers used to collect data, and (3) uncertainty over the
ultimate cost of the census, currently estimated at more than $14
billion.[Footnote 1] In the past year, the Bureau has made progress on
these challenges, in part by strengthening its risk management efforts.
Still, the census remains high risk because a critical risk management
exercise planned for 2008--a "dress rehearsal" of all census
operations--was curtailed.[Footnote 2] As a result, key operations and
systems, including some that will be used for the first time in a
census, were not tested in concert with one another or under census-
like conditions.
This year, 2009, will be one of the most crucial time periods in this
decade-long census cycle. The Bureau has already initiated large-block
canvassing--an operation where temporary field workers validate address
lists and maps for census blocks with more than 1,000 housing units in
them. Next month, the Bureau is scheduled to conduct address canvassing
for remaining census blocks when about 140,000 temporary employees will
walk every known street in the country trying to update and verify the
Bureau's address list and maps for the country. Later in the year, in a
separate effort, the Bureau is scheduled to update the locations of
approximately 200,000 "group quarters" including homeless shelters,
college residence halls, and group homes. The Bureau will also be
opening hundreds of local census offices and refining plans for later
operations.
As requested, in our remarks today, we will discuss the state of the
census, paying particular attention to the following:
* the importance of reliable cost estimates and justifications for
spending on census activities;
* building a complete and accurate address list to know where to count
people;
* following up on missing and conflicting responses to ensure
completeness and accuracy;
* targeting communications and outreach efforts to reduce the
differential undercount; and:
* designing, testing, and implementing technology to support the
census.
Because the effectiveness of these activities will determine in large
part the final cost and accuracy of the 2010 enumeration, they are
important building blocks of a successful enumeration.
Our testimony today is based on our ongoing and recently completed
work. See the last page of this statement for a list of our recently
issued census reports. To identify key issues the Bureau faces as it
approaches the 2010 Census, we reviewed and analyzed scheduling,
design, operational, and testing plans for the various census
operations, data from the dress rehearsal sites, and documents related
to the December 2008 field test of the handheld computers in
Fayetteville, North Carolina, and we interviewed Bureau staff. At the
field test, we observed the handheld computers' ability to collect and
transmit address data by accompanying census workers as they went door-
to-door. In February 2009, we also observed census workers conduct
large-block canvassing using laptop computers. We conducted our work in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audits to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions.
In summary, the Bureau has made commendable progress in rolling out key
components of the census, making improvements to the handheld
computers, certain risk management efforts, and how it will print the
80 million maps needed by temporary field staff to carry out the
enumeration. Nevertheless, at a time when planning activities should be
reaching completion, major testing should be winding down, and there
should be confidence in the functionality of census-taking activities,
the Bureau instead finds itself lacking sufficient policies,
procedures, and trained staff to develop high-quality cost estimates,
and a number of operations and support systems still need to be
designed, planned, or tested. In the 13 months leading up to Census
Day, the Bureau will be challenged to implement early operations,
complete the final preparations for various activities, make
refinements, and address any glitches that arise. With little time
remaining, uncertainties surround the Bureau's readiness for 2010.
Background:
As you know, Mr. Chairman, the decennial census is a critical national
effort mandated by the Constitution. Census data are used to apportion
seats in the Congress, redraw congressional districts, allocate
billions of dollars in federal assistance to state and local
governments, and for numerous other public and private sector purposes.
The Bureau estimates that the 2010 Census will cost more than $14
billion over its life-cycle, making it the most expensive census in our
nation's history. According to the Bureau, the increasing cost of the
census is caused in part by various societal trends--such as increasing
privacy concerns, more non-English speakers, and people residing in
makeshift and other nontraditional living arrangements--making it
harder to find people and get them to participate in the census.
In light of these challenges, it will be important for the Bureau to
draw upon the lessons learned from previous decennials and apply them
to the operational environment it faces today.
Some broad lessons learned that we have identified from our past work
that directly affect the cost and accuracy of the census include the
following:
* Sound risk management is critical to a successful census as the risks
to a cost-effective enumeration are interrelated, and a shortcoming in
one operation could trigger subsequent activities to spiral downward.
Of course the reverse is also true, where a success in one operation
could have a number of positive effects later in the process. Rigorous
up-front preparations, testing, and where feasible, contingency
planning, are the best ways to stave off problems. Likewise, management
information systems capable of tracking key operations with real-time
measures are essential because they enable the Bureau to quickly
address trouble spots.
* It is important for the Bureau to stay on schedule, as the census is
conducted against a backdrop of immutable deadlines, and an elaborate
chain of interrelated pre-and post-Census Day activities are predicated
upon those dates. Specifically, the Secretary of Commerce is legally
required to (1) conduct the census on April 1 of the decennial year,
(2) report the state population counts to the President for purposes of
congressional apportionment by December 31 of the decennial year, and
(3) send population tabulations to the states for purposes of
redistricting no later than April 1 of the year following Census Day.
To meet these reporting requirements, it is absolutely critical for the
Bureau to stay on schedule. The figure shows some planned dates for
selected decennial events.
Figure 1: Timeline of Selected Decennial Events:
[Refer to PDF for image: illustration]
Operation or activity: Local update of census addresses (LUCA):
Localities assist in updating address lists and maps;
Date: starting January 2007 through February 2010.
Operation or activity: Opening of 494 local census offices;
Date: October 2008 through December 2009.
Operation or activity: Large block canvassing: Field staff validate
addresses for blocks containing over 1,000 housing units;
Date: January 2009 through June 2009.
Operation or activity: Address canvassing: Field staff validate address
lists and maps;
Date: April 2009 through July 2009.
Operation or activity: Group quarters validation: Field staff validate
addresses for group housing such as prisons and nursing facilities;
Date: September 2009 through October 2009.
Operation or activity: Mailout/mailback: Most households are mailed
census questionnaires;
Date: March 2009 through October 2010.
April 1, 2010: Census Day.
Operation or activity: Update/enumerate: Field staff visit housing
units that do not have house numbers and/or street names;
Date: March 2010 through May 2010.
Operation or activity: Group quarters enumeration: Field staff visit
group housing such as prisons and nursing facilities;
Date: March 2010 through May 2010.
Operation or activity: Nonresponse follow-up: Field staff follow-up in
person with nonresponding households;
Date: May 2010 through July 2010.
Operation or activity: Coverage follow-up: Staff follow-up by telephone
to resolve conflicting information provided on census forms;
Date: April 2010 through August 2010.
December 31, 2010: Delivery of apportionment counts to the President.
April 1, 2011: Complete delivery of redistricting data to states.
Source: GAO summary of U.S. Census Bureau information.
[End of figure]
* Finally, the decennial census is a shared national undertaking, where
Congress; other federal agencies; state, local, and tribal governments;
nonprofit and private organizations; and, ultimately, the American
public, all play vital roles in securing a complete and accurate
population tally. Recognizing this, the Bureau fosters partnerships
with these various entities to help with such activities as recruiting
census workers, boosting participation, and building the Bureau's
master address list. Mobilizing and coordinating these organizations
requires an enormous effort on the Bureau's part.
Providing Reliable Cost Estimates and Justifications for Spending as
2010 Approaches Presents a Major Challenge for the Bureau:
Accurate cost estimates are essential to a successful census because
they help ensure that the Bureau has adequate funds, and so that
Congress, the administration, and the Bureau itself can have reliable
information on which to base or advise decisions. However, as we have
reported before, the Bureau has insufficient policies and procedures
and inadequately trained staff for conducting high-quality cost
estimation for the decennial census.[Footnote 3] The Bureau does not
have cost estimation guidance and procedures in place or staff that is
certified in cost estimation techniques. The Bureau is developing a new
budget management tool that will support the cost estimation process
beyond 2010. As part of that effort, the Bureau will need to establish
rigorous cost estimation policies and procedures and use skilled
estimators to ensure that future cost estimates are reliable and of
high quality.
For example, to help manage the 2010 Census and contain costs, over 5
years ago we recommended that the Bureau develop a comprehensive,
integrated project plan for the 2010 Census that should include the
itemized, estimated costs of each component and a sensitivity analysis
[Footnote 4] and an explanation of significant changes in the
assumptions on which these costs were based.[Footnote 5] In response,
the Bureau provided us with the 2010 Census Operations and Systems
Plan, dated August 2007. This plan represented an important step
forward by including operational inputs and outputs and describing
linkages among operations and systems. However, that document did not
include itemized cost estimates of each component or sensitivity
analyses, and thus did not provide a valid baseline or range of
estimates for the Bureau and Congress. The Bureau has provided annual
cost updates as part of its budget submission process, but these too
have lacked cost analyses to support them. As the Bureau approaches the
final surge in the current decade-long decennial spending cycle,
providing reliable cost estimates accompanied by sound justification,
as we have recommended, will be important if Congress is to make
informed decisions on the levels at which to fund the remainder of the
2010 Decennial Census.
Effective Address Canvassing Is Essential for a Complete and Accurate
Count:
A complete and accurate list of all addresses where people live in the
country is the cornerstone of a successful census because it identifies
all households that are to receive a census questionnaire and serves as
the control mechanism for following up with households that fail to
respond. The Bureau goes to great lengths to develop a quality address
list and maps, working with the U.S. Postal Service; federal agencies;
state, local, and tribal governments; local planning organizations;
private sector; and nongovernmental entities. For example, under the
Local Update of Census Addresses (LUCA) program, the Bureau is
authorized to partner with state, local, and tribal governments,
tapping into their knowledge of local populations and housing
conditions in order to secure a more complete count.[Footnote 6]
Between November 2007 and March 2008, over 8,000 state, local, and
tribal governments provided approximately 8 million address updates
through the LUCA program. The Bureau will send thousands of temporary
census workers, known as listers, into the field to collect and verify
address information and update maps on-site, including verifying
address updates provided through the LUCA program.
Despite the Bureau's efforts, an inherent challenge is locating
unconventional and hidden housing units, such as converted basements
and attics. For example, as shown in figure 2, what appears to be a
small, single-family house could contain an apartment, as suggested by
its two doorbells. The Bureau has trained listers to look for extra
mailboxes, utility meters, and other signs of hidden housing units and
is developing training guides for 2010 to help listers locate hidden
housing. Nonetheless, decisions on what is a habitable dwelling are
often difficult to make--what is habitable to one worker may seem
uninhabitable to another. According to Bureau estimates, approximately
1.4 million housing units were missed in the 2000 Census. If an address
is not in the Bureau's address file, its residents are less likely to
be included in the census.
Figure 2: Single or Multi-unit Housing?
[Refer to PDF for image: photograph]
Photograph of single family home. Two doorbells could indicate two
housing units.
Source: GAO.
[End of figure]
Performance of Handheld Computers Have Improved in Field Testing, but
More Information Is Needed to Evaluate Readiness for Address
Canvassing:
A nationwide address canvassing operation for the 2010 Census is
scheduled to begin this spring, when listers will use handheld
computers for the first time to collect address data. Listers will add
addresses that do not already appear on the Bureau's list and mark for
deletion any that they cannot verify according to the rules and
guidance developed by the Bureau.
When the handheld computers were tested during the dress rehearsal of
the address canvassing operation, the devices experienced such problems
as slow or inconsistent data transmission, freeze-ups, and difficulties
collecting mapping coordinates.[Footnote 7] The software that managers
used to review work productivity and assign work was also troublesome.
[Footnote 8] For example, management reports were unreliable because
they pulled data from incorrect sources, and Bureau staff had
difficulty using the work management software to reassign work.
The Bureau took steps to fix these issues, and in December 2008
conducted a limited field test in Fayetteville, North Carolina, to test
the functionality and usability of the handheld computer, including
whether the handheld computer problems encountered earlier had been
resolved. Although the Bureau's final evaluation of the field test was
due by the end of February 2009, we were not able to review it for this
testimony. From observations of the December 2008 field test and
interviews with Bureau officials, the Bureau appears to have addressed
many of the handheld computer performance issues, as well as the
problems with the work management software, observed during the dress
rehearsal. This is an important and noteworthy development.
Nonetheless, more information is needed to determine the Bureau's
overall readiness for address canvassing as the field test was not an
end-to-end systems evaluation, did not validate all address canvassing
requirements, such as training and help desk support, and did not
include urban areas. Additionally, the scale of the field test was a
fraction of that of the address canvassing operation. The Bureau was to
conduct a review of the readiness of the handheld computers in January
2009, but has not yet reported the results of that review. Finally, the
Bureau's actual workload for address canvassing--about 144.7 million
addresses--is 11 million addresses more than the Bureau had planned
for, leaving the Bureau with too few handheld computers to complete the
workload in the time originally scheduled. In response, the Bureau will
be extending the amount of time listers will be working in the field in
affected areas, although not extending the end date of the operation,
to compensate for the larger workload.
During dress rehearsal, listers experienced problems using handheld
computers when collecting address data for large blocks having more
than 1,000 housing units. According to the Bureau, the handheld
computer did not have the capacity to efficiently collect data for
large blocks. The Bureau has taken steps to mitigate this problem. In
January 2009, the Bureau began using laptop computers and software
already used in other operations to canvass the 2,086 blocks it
identified as large blocks and by the end of February, the Bureau had
completed approximately 80 percent of large block canvassing.[Footnote
9] In February 2009, we observed large-block canvassing in Atlanta,
Georgia; Boston, Massachusetts; New York, New York; San Francisco,
California; and Washington, D.C. Based on our preliminary observations,
the laptops appear to work well and listers reported their training was
satisfactory. We are in the process of discussing these and other
observations with the Bureau.
Bureau Needs to Finalize Field Data Collection Plans:
The Bureau's largest and most costly field operation is nonresponse
follow-up. The Bureau estimates that it will employ over 600,000
temporary workers to collect data from about 47 million nonrespondening
households over the course of 10 weeks in 2010. On April 3, 2008, the
Bureau announced that it would no longer use handheld computers for
nonresponse follow-up and would instead change to a paper-based
nonresponse follow-up operation. According to the Bureau, this change
added between $2.2 billion to $3 billion to the total cost of the
census.
In May 2008, the Bureau issued a plan that covered major components of
the paper-based nonresponse follow-up. Bureau officials said that they
are developing a more detailed plan that would describe 2010
nonresponse follow-up operations and systems, workflow, major
milestones, and roles and responsibilities of different census
divisions. Although the plan was due in January 2009, it has yet to be
completed. Because this plan serves as a road map for monitoring the
development and implementation of nonresponse follow-up, it will be
important for the Bureau to complete this plan.
The Bureau has changed plans for many aspects of nonresponse follow-up,
and officials are determining which activities and interfaces will be
tested and when this testing will occur. Although the Bureau has
carried out a paper-based follow-up operation in past decennials, the
2010 Census includes new procedures and system interfaces that have not
been tested under census-like conditions because they were dropped from
the dress rehearsal. Bureau officials acknowledged the importance of
testing new and modified nonresponse follow-up activities and system
interfaces to reduce risk, but have not yet developed detailed testing
plans. Given the number of tasks at hand and the increasingly shorter
time frame in which to accomplish them, it will be important for the
Bureau to monitor the development of these testing plans, coordinate
this testing with other activities, and ensure that testing occurs in
time to take corrective actions, if needed.
In our previous work, we have highlighted the importance of sound risk
management in planning for the decennial census.[Footnote 10] The
Bureau has strengthened aspects of its risk management process. For
example, in July 2008, the Bureau identified 31 nonresponse follow-up
risks, such as lower than expected enumerator productivity. However, it
has not developed mitigation plans for these risks. Officials said that
they are reevaluating these risks and plan to develop mitigation plans
for high-and medium-priority nonresponse follow-up risks starting in
spring 2009. However, the Bureau has not yet determined when these
plans will be completed.
Coverage Follow-up Operation Needs to Be Finalized:
One of the Bureau's long-standing challenges is resolving conflicting
information respondents provide on census forms. This problem can
occur, for example, when the number of household members reported on a
completed form differs from the number of persons for whom information
is provided. In such instances, the Bureau attempts to reconcile the
data during the coverage follow-up operation. For 2010, the Bureau
plans to expand the scope of this operation and include two questions-
-known as coverage probes--on the census form to identify households
where someone may have been missed or counted incorrectly (see fig. 3).
Figure 3: Example of Coverage Probes from Draft 2010 Census Form:
[Refer to PDF for image: illustration]
Undercount probe:
2. Were there any additional people staying here April 1, 2010 that you
did not include in Question 1? Mark all that apply:
Children, such as new born babies or foster children;
Relatives, such as adult children, cousins, or in-laws;
People staying here temporarily;
No additional people.
Overcount probe:
10. Does person 1 sometimes live or stay somewhere else?
No:
Yes: Mark all that apply:
In college housing;
In the military;
At a seasonal or second residence;
For child custody;
In jail or prison;
In a nursing home;
For another reason.
Source: GAO presentation of U.S. Census Bureau information.
[End of figure]
However, after testing the probes earlier in the decade, the Bureau
found that one of the probes was problematic in identifying persons
potentially missing from the count. Although these probes were included
on the forms mailed out during the dress rehearsal, the coverage follow-
up operation did not include cases from nonresponse follow-up, which
was canceled from the dress rehearsal. In the absence of a final test
of the coverage probes in nonresponse follow-up, the effectiveness of
the information generated by the probes is uncertain.
Fieldwork Management System for Most Operations Still Needs to Be
Specified and Programmed:
A successful census depends, in large part, on the work carried out in
the local census offices.[Footnote 11] For the 2010 Census, this field
work cannot be accomplished without a properly functioning Operations
Control System (OCS). This system is intended to provide managers with
essential real-time information, such as worker productivity and
completion rates for field operations. It also allows managers to
assign or reassign cases among workers. If the system does not work as
intended, it could bog down or delay field operations and introduce
errors into data collected.
Initially, the Bureau had planned to use a contractor to develop OCS to
manage the workflow for those operations relying on paper-based
processes, such as group quarters enumeration and nonresponse follow-
up. However, in August 2008, the Bureau created an internal program to
develop OCS and other related infrastructure that are needed to support
these operations. The Bureau is still in the process of developing OCS
for paper-based operations.
Although the Bureau has established a high-level schedule for testing
of OCS, it has not yet finalized the requirements needed to begin its
programming or developed a detailed schedule for conducting additional
tests. Further, the Bureau has not yet fully defined how OCS will work
together with other systems. Bureau officials said that the lack of
detailed plans for operations, such as nonresponse follow-up, makes it
difficult to finalize requirements for OCS or its testing plans. Our
work on IT systems testing has shown that without adequate oversight
and more comprehensive guidance, the Bureau cannot ensure that it is
thoroughly testing its systems and properly prioritizing testing
activities before the 2010 Census.
The Bureau Has Taken Steps to Improve Map Production, but Faces a Tight
Schedule:
The Bureau estimates that it will need to produce approximately 30
million different map files from which 80 million paper maps will be
printed to assist census workers in locating addresses in major census
operations. The quality of maps and the timing of map printing are
critical to the success of the census. In addition, many map production
and printing activities must be conducted in sequence with no time to
spare, putting at risk the Bureau's ability to print its maps on time.
The Bureau has taken positive steps to meet its requirements for map
production and printing for 2010. For example, in June 2008, the Bureau
decided to produce a generic map type in lieu of several operation-
specific versions to reduce the number of map files to be produced.
Furthermore, the Bureau is preparing to print most of its maps at the
local census offices rather than at the regional offices, reducing the
need to coordinate map delivery to the local census offices. In
addition, the Bureau has replaced its labor-intensive quality assurance
process with integrated, automated processes. These steps taken to
improve workflow will become particularly important as the Bureau works
to produce and print maps on an already compressed schedule.
The Bureau's schedule for producing and printing maps does not allow
for any delays in receiving data from other operations or from the
contractor delivering map files. For example, the Bureau intends to
include map information from address canvassing, which ends in July
2009, in maps that will be used to validate locations of group
quarters, which begins in September 2009. Bureau officials have stated
that the turnaround time between these operations allows no slippage,
and if these data are received late, an entire chain of subsequent map
production steps would be thrown off schedule. Furthermore, according
to the Bureau, local census offices need to receive map files from the
contractor in time to print maps for certain field operations by
January 8, 2010. However, the contractor is not scheduled to finish
delivering the map files until January 19, 2010. Bureau officials said
that they have taken steps to ensure that the necessary map files are
delivered in time for printing but are still working to resolve the
discrepancy.
Census Marketing Programs Will Need to Improve Response Rates of
Historically Undercounted Groups:
The Bureau goes to great lengths to reduce the undercount, especially
among those groups likely to be undercounted at a higher rate than
others, such as minorities and renters. For example, the Bureau plans
to provide language assistance guides in 59 languages for completing
the census, an increase from 49 languages in 2000. For the first time
in 2010, the Bureau plans to send bilingual questionnaires to
approximately 13 million households that are currently likely to need
Spanish language assistance, as determined by analyzing recent data
from a related Bureau survey program.
The Bureau also plans to deploy a multifaceted communications campaign
consisting of, among other efforts, paid advertising and the hiring of
as many as 680 partnership staff who will be tasked with reaching out
to local governments, community groups, and other organizations in an
effort to secure a more complete count. Overall, the Bureau estimates
it will spend around $410 million on its communication efforts for the
2010 Census;. However, in constant 2010 dollars, this amount is
somewhat less than the approximately $480 million that the Bureau spent
marketing the 2000 Census.
Although the effects of the Bureau's communication efforts are
difficult to measure, the Bureau reported some positive results from
its 2000 Census marketing efforts with respect to raising awareness of
the census. For example, four population groups--non-Hispanic Blacks,
non-Hispanic Whites, Asians, and Native Hawaiians--indicated they were
more likely to return the census form after the 2000 Census partnership
and marketing program than before its onset. However, a Bureau
evaluation demonstrated only a limited linkage between the partnership
and marketing effort and improvements in actual census mail return
behavior for these or other groups. Put another way, while the Bureau's
marketing activities might raise awareness of the census, a remaining
challenge is converting that awareness into an actual response. Other
marketing challenges include long-standing issues such as the nation's
linguistic diversity and privacy concerns, as well as a number of newly
emerging concerns such as local campaigns against illegal immigration
and a post-September 11 environment that could heighten some groups'
fears of government agencies.
Managing and Testing of Information Technology Systems Remain a
Concern:
Since 2005, we have reported on weaknesses in the Bureau's management
of its IT acquisitions, and we remain concerned about the Bureau's IT
management and testing of key 2010 Census systems. For example, in
October 2007, we reported on the status of and plans for key 2010
Census IT acquisitions and whether the Bureau was adequately managing
associated risks.[Footnote 12] We found critical weaknesses in the
Bureau's risk management practices, including those associated with
risk identification, mitigation, and oversight. We later presented
multiple testimonies on the Bureau's progress in addressing significant
risks facing the 2010 Census. In particular, the Field Data Collection
Automation (FDCA) program, which includes the development of handheld
computers for the address canvassing operation and the systems,
equipment, and infrastructure that field staff will use to collect
data, has experienced significant problems. For example, in March 2008,
we testified that the FDCA program was experiencing schedule delays and
cost increases, and was contributing significant risk to the 2010
Census. At that time, we highlighted our previous recommendations to
better manage FDCA and the other IT acquisitions.[Footnote 13]
In response to our findings and recommendations, the Bureau has taken
several steps to improve its management of IT for the 2010 Census. For
example, the Bureau has sought external assessments of its activities
from independent research organizations, implemented a new management
structure and management processes and brought in experienced personnel
to key positions, and improved several reporting processes and metrics.
In part, due to our review of the FDCA program, the Bureau requested a
revised cost proposal for the FDCA program which resulted in a cost
reduction of about $318 million for the remaining 5-year life-cycle of
the program.
As we have previously reported, operational testing planned during the
census dress rehearsal would take place without the full complement of
systems and functionality that was originally planned, and it was
unclear whether the Bureau was developing plans to test all
interrelated systems and functionality. At your request, we reviewed
the status and plans of testing of key 2010 Census systems. As stated
in our report, which we are releasing today, we found that the Bureau
has made progress in conducting systems, integration, and end-to-end
testing, but critical testing still remains to be performed before
systems will be ready to support the 2010 Census, and the planning,
execution, and monitoring of its testing needs much improvement.
[Footnote 14] We are making 10 recommendations for strengthening the
Bureau's testing of 2010 Census systems. Those recommendations address
improvements needed in test planning, management, and monitoring. In
response to our report, the Department of Commerce and Bureau stated
they had no significant disagreements with our recommendations.
Concluding Observations:
In summary, little more than a year remains until Census Day. At a time
when major testing should be completed and there should be confidence
in the functionality of key operations, the Bureau instead finds itself
managing late design changes and developing testing plans. The Bureau
has taken some important steps toward mitigating some of the challenges
that it has faced to date, yet much remains uncertain, and the risks to
a successful decennial census remain.
Addressing these risks and challenges will be critical to the timely
completion of a cost-effective census, and it will be essential for the
Bureau to develop plans for testing systems and procedures not included
in the dress rehearsal, and for Congress to monitor the Bureau's
progress.
As always, we look forward to working with Congress in assessing the
Bureau's efforts to overcome these hurdles to a successful census, and
providing regular updates on the rollout of the decennial in the
critical months that lie ahead.
Mr. Chairman and members of this Subcommittee, this concludes our
statement. We would be happy to respond to any questions that you or
members of the Subcommittee may have at this time.
If you have any questions on matters discussed in this testimony,
please contact Robert Goldenkoff at (202) 512-2757 or David A. Powner
at (202) 512-9286 or by e-mail at goldenkoffr@gao.gov or
pownerd@gao.gov. Other key contributors to this testimony include
Sher'rie Bacon, Thomas Beall, Steven Berke, Vijay D'Souza, Elizabeth
Fan, Richard Hung, Andrea Levine, Signora May, Ty Mitchell, Catherine
Myrick, Lisa Pearson, Kathleen Padulchick, Crystal Robinson, Melissa
Schermerhorn, Cynthia Scott, Karl Seifert, Jonathan Ticehurst, Timothy
Wexler, and Katherine Wulff.
[End of section]
Related GAO Products:
High-Risk Series: An Update. [hyperlink,
http://www.gao.gov/products/GAO-09-27]. Washington, D.C.: January 2009.
2010 Census: The Bureau's Plans for Reducing the Undercount Show
Promise, but Key Uncertainties Remain. [hyperlink,
http://www.gao.gov/products/GAO-08-1167T]. Washington, D.C.: September
23, 2008.
2010 Census: Census Bureau's Decision to Continue with Handheld
Computers for Address Canvassing Makes Planning and Testing Critical.
[hyperlink, http://www.gao.gov/products/GAO-08-936]. Washington, D.C.:
July 31, 2008.
2010 Census: Census Bureau Should Take Action to Improve the
Credibility and Accuracy of Its Cost Estimate for the Decennial Census.
[hyperlink, http://www.gao.gov/products/GAO-08-554]. Washington, D.C.:
June 16, 2008.
2010 Census: Plans for Decennial Census Operations and Technology Have
Progressed, But Much Uncertainty Remains. [hyperlink,
http://www.gao.gov/products/GAO-08-886T]. Washington, D.C.: June 11,
2008.
2010 Census: Bureau Needs to Specify How It Will Assess Coverage Follow-
up Techniques and When It Will Produce Coverage Measurement Results.
[hyperlink, http://www.gao.gov/products/GAO-08-414]. Washington, D.C.:
April 15, 2008.
2010 Census: Census at Critical Juncture for Implementing Risk
Reduction Strategies. [hyperlink,
http://www.gao.gov/products/GAO-08-659T]. Washington, D.C.: April 9,
2008.
Information Technology: Significant Problems of Critical Automation
Program Contribute to Risks Facing 2010 Census. [hyperlink,
http://www.gao.gov/products/GAO-08-550T]. Washington, D.C.: March 5,
2008.
Information Technology: Census Bureau Needs to Improve Its Risk
Management of Decennial Systems. [hyperlink,
http://www.gao.gov/products/GAO-08-259T]. Washington, D.C.: December
11, 2007.
2010 Census: Planning and Testing Activities Are Making Progress.
[hyperlink, http://www.gao.gov/products/GAO-06-465T]. Washington, D.C.:
March 1, 2006.
[End of section]
Footnotes:
[1] GAO, Information Technology: Significant Problems of Critical
Automation Program Contribute to Risks Facing 2010 Census, [hyperlink,
http://www.gao.gov/products/GAO-08-550T] (Washington, D.C.: Mar. 5,
2008).
[2] GAO, High-Risk Series: An Update, [hyperlink,
http://www.gao.gov/products/GAO-09-271] (Washington, D.C.: Jan. 2009).
[3] GAO, 2010 Census: Census Bureau Should Take Action to Improve the
Credibility and Accuracy of Its Cost Estimate for the Decennial Census,
[hyperlink, http://www.gao.gov/products/GAO-08-554] (Washington, D.C.:
Jun. 15, 2008).
[4] Sensitivity analysis examines the effect of changing one assumption
or cost driver at a time while holding all other variables constant.
[5] GAO, 2010 Census: Cost and Design Issues Need to Be Addressed Soon,
[hyperlink, http://www.gao.gov/products/GAO-04-37] (Washington, D.C.:
Jan. 15, 2004).
[6] Census Address List Improvement Act of 1994, Pub. L. No. 103-430.
[7] GAO, 2010 Census: Census Bureau's Decision to Continue with
Handheld Computers for Address Canvassing Makes Planning and Testing
Critical, [hyperlink, http://www.gao.gov/products/GAO-08-936]
(Washington, D.C.: Jul. 31, 2008).
[8] GAO, 2010 Census: Plans for Decennial Census Operations and
Technology Have Progressed, But Much Uncertainty Remains, [hyperlink,
http://www.gao.gov/products/GAO-08-886T] (Washington, D.C.: Jun. 11,
2008).
[9] These 2,086 large blocks are located in 332 counties and are
concentrated in the following regions: Atlanta, Charlotte, Dallas,
Denver, Los Angeles, and New York.
[10] GAO, 2010 Census: Plans for Decennial Census Operations and
Technology Have Progressed, But Much Uncertainty Remains, [hyperlink,
http://www.gao.gov/products/GAO-08-886T] (Washington, D.C.: Jun. 11,
2008).
[11] For all decennial census operations, the Bureau plans to hire 1.4
million temporary employees who will receive their training and work
assignments through 494 local census offices, as well as the 12
regional census centers throughout the country.
[12] GAO, Information Technology: Census Bureau Needs to Improve Its
Risk Management of Decennial Systems, [hyperlink,
http://www.gao.gov/products/GAO-08-79] (Washington, D.C.: Oct. 5,
2007).
[13] [hyperlink, http://www.gao.gov/products/GAO-08-550T].
[14] GAO, Information Technology: Census Bureau Testing of 2010
Decennial Systems Can Be Strengthened, [hyperlink,
http://www.gao.gov/products/GAO-09-262] (Washington, D.C.: Mar. 5,
2009).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: