2010 Census
Cost and Design Issues Need to Be Addressed Soon Gao ID: GAO-04-37 January 15, 2004The key to a successful census is meticulous planning as it helps ensure greater cost-effectiveness. However, the 2000 and previous censuses have been marked by poor planning, which unnecessarily raised the costs and risks of those efforts. GAO was asked to (1) review the U.S. Census Bureau's (Bureau) current plans for 2010 and whether they might address shortcomings of the 2000 Census, (2) analyze the Bureau's cost estimates, and (3) review the rigor of the Bureau's 2010 planning process.
While preparations for the 2010 Census appear to be further along compared to a similar point prior to the 2000 Census, cost and design information had to be pieced together from various documents. The Bureau's plans also lack a substantial amount of supporting analysis, budgetary transparency, and other information that made it difficult to verify the Bureau's assertions concerning the costs and benefits of its proposed approach. Further, unlike in previous censuses, the Bureau does not intend to develop coverage measurement procedures that would allow it to adjust census data for certain purposes. Although its experience in 2000 shows that its coverage measurement methodology needs improvement, GAO believes the Bureau should have researched alternative approaches more thoroughly and disclosed the results of its research before making a decision. In designing the 2010 Census, the Bureau hoped to address several shortcomings of the 2000 enumeration, namely to (1) increase the relevance and timeliness of data, (2) reduce operational risk, (3) increase coverage and accuracy, and (4) contain costs. To achieve these goals, three components--all new operations--are key to the Bureau's plans for 2010. They include enhancing procedures for building the census address list and associated maps, replacing the census long-form questionnaire with a more frequent sample survey, and conducting a short-form-only census. The Bureau's approach has the potential to achieve the first three goals, but reducing operational risk could prove to be more difficult as each of the three components actually introduces new risks. The Bureau will also be challenged to control the cost of the 2010 Census, now estimated at over $11 billion. The current budget reporting process masks the long-term costs of the census, most of which will be incurred in 2010; making it difficult for Congress to monitor the Bureau's planned expenditures. Certain actions by the Office of Management and Budget could produce greater fiscal transparency, and thus help inform congressional deliberations on how to best balance the need for an accurate census, with the need to ensure a reasonable cost for this endeavor.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-37, 2010 Census: Cost and Design Issues Need to Be Addressed Soon
This is the accessible text file for GAO report number GAO-04-37
entitled '2010 Census: Cost and Design Issues Need to Be Addressed
Soon' which was released on January 28, 2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
January 2004:
2010 CENSUS:
Cost and Design Issues Need to Be Addressed Soon:
GAO-04-37:
GAO Highlights:
Highlights of GAO-04-37, a report to congressional requesters
Why GAO Did This Study:
The key to a successful census is meticulous planning as it helps
ensure greater cost-effectiveness. However, the 2000 and previous
censuses have been marked by poor planning, which unnecessarily
raised the costs and risks of those efforts. GAO was asked to (1)
review the U.S. Census Bureau‘s (Bureau) current plans for 2010 and
whether they might address shortcomings of the 2000 Census, (2)
analyze the Bureau‘s cost estimates, and (3) review the rigor of the
Bureau‘s 2010 planning process.
What GAO Found:
While preparations for the 2010 Census appear to be further along
compared to a similar point prior to the 2000 Census, cost and design
information had to be pieced together from various documents. The
Bureau‘s plans also lack a substantial amount of supporting analysis,
budgetary transparency, and other information that made it difficult
to verify the Bureau‘s assertions concerning the costs and benefits of
its proposed approach. Further, unlike in previous censuses, the
Bureau does not intend to develop coverage measurement procedures that
would allow it to adjust census data for certain purposes. Although
its experience in 2000 shows that its coverage measurement methodology
needs improvement, GAO believes the Bureau should have researched
alternative approaches more thoroughly and disclosed the results of
its research before making a decision. In designing the 2010 Census,
the Bureau hoped to address several shortcomings of the 2000
enumeration, namely to (1) increase the relevance and timeliness of
data, (2) reduce operational risk, (3) increase coverage and accuracy,
and (4) contain costs. To achieve these goals, three components”all
new operations”are key to the Bureau‘s plans for 2010. They include
enhancing procedures for building the census address list and
associated maps, replacing the census long-form questionnaire with a
more frequent sample survey, and conducting a short-form-only census.
Census Costs Are Increasing (Average Cost per Housing Unit in Constant
Fiscal 2000 Dollars):
[See PDF for image]
[End of figure]
The Bureau‘s approach has the potential to achieve the first three
goals, but reducing operational risk could prove to be more difficult
as each of the three components actually introduces new risks. The
Bureau will also be challenged to control the cost of the 2010 Census,
now estimated at over $11 billion. The current budget reporting
process masks the long-term costs of the census, most of which will be
incurred in 2010; making it difficult for Congress to monitor the
Bureau‘s planned expenditures. Certain actions by the Office of
Management and Budget could produce greater fiscal transparency, and
thus help inform congressional deliberations on how to best balance
the need for an accurate census, with the need to ensure a reasonable
cost for this endeavor.
What GAO Recommends:
GAO recommends that the Secretary of Commerce direct the Bureau to
combine information on goals, costs, and other key planning elements
into a single integrated project plan that is updated as needed. The
Bureau should also study the feasibility of procedures that could
allow it to adjust census results for those purposes where it is both
legal and appropriate to do so.
Further, the Director of the Office of Management and Budget should
ensure that the Bureau take steps to improve the transparency of
census life cycle costs. In commenting on a draft of this report, both
agencies disagreed with our interpretation of key findings. They also
disagreed with our recommendations. We believe our findings and
recommendations still apply.
www.gao.gov/cgi-bin/getrpt?GAO-04-37.
To view the full product, including the scope and methodology, click
on the link above. For more information, contact Patricia Dalton at
(202) 512-6806 or daltonp@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Scope and Methodology:
2010 Design Has Potential, but Introduces New Risks:
Escalating Census Costs Call for Transparent Budget Data and Better
Reporting:
2010 Planning Process Is Better Than Past Efforts, but Rigor Is
Uneven:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Comments from the Department of Commerce:
Appendix II: Comments from the Office of Management and Budget:
Appendix III: 2010 Census Planning Organization:
Tables:
Table 1: The Three-Legged Stool Strategy Could Help the Bureau Progress
toward Its Key Goals for the 2010 Census:
Table 2: Projected Costs from June 2001 Estimate of Reengineering the
2010 Census:
Figures:
Figure 1: Decennial Census Costs from 1970 through 2010 (Projected) in
Constant 2000 Dollars:
Figure 2: Decennial Census Average Cost per Housing Unit (in Constant
Fiscal Year 2000 Dollars):
Figure 3: 2000 Census Life Cycle Costs by Fiscal Year:
Figure 4: Bureau's June 2001 Estimating Methodology for Census 2010 Life
Cycle Costs:
Figure 5: Decennial Management Division's Organizational Structure for
Planning the 2010 Census:
Letter January 15, 2004:
The Honorable Wm. Lacy Clay:
Ranking Minority Member:
Subcommittee on Technology, Information Policy, Intergovernmental
Relations and the Census:
Committee on Government Reform:
House of Representatives:
The Honorable Danny K. Davis:
The Honorable Charles A. Gonzales:
The Honorable Carolyn B. Maloney:
House of Representatives:
In designing the 2010 Census, the U.S. Census Bureau (Bureau) faces the
daunting challenge of cost effectively counting a population that is
growing steadily larger, more diverse, and increasingly difficult to
enumerate, with a design that meets the sometimes competing
requirements and interests of a vast spectrum of stakeholders that
include Congress, government agencies, the statistical community,
advocacy groups, and the public.
Planning a successful census--one that is acceptable to Congress and
other stakeholders and achieves desired outcomesærequires the Bureau to
assess the lessons learned from past enumerations, identify initiatives
that show promise for producing a better census while controlling
costs, test the feasibility of these initiatives, and convince
stakeholders of the value of the proposed plans.[Footnote 1] However,
the Bureau's planning efforts for the 2000 and earlier censuses
contained a number of weaknesses that unnecessarily increased the cost
and risk of those efforts.
In January 2003, we named the 2010 Census a major performance and
accountability challenge because of our growing concern over the
numerous obstacles to a cost-effective head count. Already, current
estimates of the cost of the 2010 Census exceed $11 billion. You asked
us to examine the decision-making process for the 2010 Census. As
agreed with your offices, our objectives for this report were to review
the Bureau's current plans for the 2010 Census and the extent to which
they might address shortcomings with the 2000 Census, analyze the
Bureau's cost estimates, and assess the rigor of the Bureau's 2010
planning process.
Results in Brief:
The Bureau's preparations for the 2010 Census appear to be further
along than at a similar point during the planning cycle for the 2000
Census. Still, considerable risks and uncertainties lie ahead. This
does not necessarily mean that the Bureau's design is flawed. Indeed,
the difficulties in conducting a successful head count call for the
Bureau to consider bold and innovative initiatives, and these are not
risk free. However, the uncertainties could jeopardize a cost-effective
head count if not addressed soon.
In designing the 2010 Census, the Bureau had four goals in mind, based
on its desire to address shortcomings with the 2000 enumeration: (1)
increase the relevance and timeliness of data, (2) reduce operational
risk, (3) increase coverage and accuracy, and (4) contain costs. To
achieve these goals, three components--all new operations--are key to
the Bureau's plans for 2010:
* enhancing procedures for building its address list, known as the
Master Address File (MAF), and its associated geographic information
system, called the Topologically Integrated Geographic Encoding and
Referencing (TIGER) database;[Footnote 2]
* replacing the census long-form questionnaire with the American
Community Survey (ACS); and:
* conducting a short-form-only decennial census supported by early
research and testing.
The Bureau's approach has the potential to achieve all but its goal to
reduce operational risk. However, this is no minor shortcoming as the
Bureau's 2010 design actually introduces new risks as the Bureau will
be challenged to (1) secure early congressional agreement on its
design, (2) resolve methodological and other uncertainties surrounding
ACS, and (3) successfully implement all three components of the
Bureau's design. Because the three components are interrelated (e.g.,
the ACS is critical to a short-form-only census), failure to
successfully carry out any one component of the plan could doom the
success of the other two, thereby reducing the overall cost-
effectiveness of the census.
A fourth challenge will be slowing the mushrooming cost of the 2010
Census, now estimated at more than $11.3 billion in current dollars.
Put another way, it will cost the nation an estimated $72 to count each
household in 2010, compared to $56 in 2000 and $13 in 1970 (in constant
2000 dollars), in large part because it is getting increasingly
difficult to locate people and get them to participate in the census.
The growing cost of the head count, at a time when the nation is facing
historic budget deficits, highlights the importance of congressional
deliberations on the extent to which each additional dollar spent on
the census results in better data, as well as how best to balance the
need for a complete count, with the need to ensure the cost of a
complete count does not become unreasonable.
In a departure from previous decennial censuses, the Bureau has decided
against developing coverage measurement procedures that would allow it
to adjust 2010 Census data for certain purposes. Although its
experience during the 2000 Census shows that its approach to measuring
coverage needs to be improved, we believe the Bureau should have
researched alternative methods more thoroughly and disclosed the
results of its research before making its decision.
Current budget reporting masks the long-term cost implications of the
census, and this could hamper or delay discussions concerning cost
containment alternatives because annual budget requests and information
provided to Congress early in the decennial life cycle do not reflect
the full cash consequences of the annual spending. Most of the early
spending will be on planning and development costs, but most of the
expenses will be incurred in 2010. As Congress begins funding the 2010
Census at relatively low levels, it will have implicitly accepted a
future spike in costs without requiring the Bureau to define more
clearly what those costs might be, why they are justified, and what
alternatives might exist. Greater fiscal transparency, including more
robust information on the Bureau's underlying budget assumptions, could
help policymakers control escalating census costs.
In some respects, the Bureau's planning for the 2010 Census appears to
be more rigorous than that for the 2000 Census. For example, the Bureau
has involved experienced staff in the design process, organized
multidivision planning groups, assigned roles and responsibilities, and
taken steps to address issues such as the attrition of key experienced
staff. Still, other aspects of a rigorous planning process were
lacking. They include (1) a comprehensive project plan with information
on operational milestones, measurable performance goals, and costs and
(2) productivity and other data on the results of key operations from
the 2000 Census to inform lessons learned for 2010.
To improve the 2010 Census planning process, we recommend that the
Secretary of Commerce direct the Bureau to consolidate information on
goals, performance measures, costs, and other key planning elements
into a single project plan that would be updated as needed. Further, to
help ensure the nation has at its disposal the best possible data
should there be problems with the quality of the census numbers, the
Secretary should also direct the Bureau, with input from both majority
and minority parties in the Senate and House of Representatives, to
research the feasibility of procedures that could allow the Bureau to
adjust census results for those purposes for which it is both legal and
appropriate to do so and, if found to be feasible, test those
procedures during the 2006 census test and 2008 census dress rehearsal.
In addition, to highlight for Congress the long-term fiscal exposure
associated with the census, signal emerging cost issues, and prompt
congressional deliberations on affordability and trade-offs, we
recommend that the Director of the Office of Management and Budget
(OMB) ensure that the Bureau take steps to increase the transparency of
census life cycle costs. Specific steps include displaying such
information as updated estimates of census life cycle costs and the
amount of money the Bureau expects to spend in each year of the cycle
as a note in the Program and Financing schedule of the President's
budget. The information should also explain any material changes from
previous budgets, the sensitivity of the cost figures to specific
assumptions, and the likelihood of those assumptions.
The Secretary of Commerce forwarded written comments from the U.S.
Census Bureau on a draft of this report. The comments are reprinted in
appendix I. The Bureau agreed that to bring closure on issues relevant
to the 2010 Census, it was important to have informed discussions with
stakeholders, especially Congress, but disagreed with the
recommendations we believe are necessary for such discussions to be
fruitful. However, based on our evaluation of the Bureau's comments, we
believe all of our recommendations still apply.
For example, the Bureau took exception to our recommendation to improve
the rigor of its planning process by developing a consolidated project
plan, noting that existing documents already provide this information.
We believe our recommendation is still pertinent because, while we
agree with the Bureau that some of this information is already
available, it is disjointed and one can only obtain it by piecing
together several different documents, such as the Bureau's budget
submission and strategic plan. And even then, it would still lack such
important information as performance goals.
The Director of OMB also prepared written comments on a draft of this
report that are reprinted in appendix II. OMB shared our view that the
costs and risks associated with the 2010 Census must be carefully
monitored and evaluated throughout the decade. OMB also agreed that it
is essential to understand the key cost drivers and said that it is
working with the Census Bureau to ensure the Bureau develops high-
quality, transparent life cycle cost estimates. However, OMB disagreed
with our recommendation that it highlight for Congress the long-term
fiscal exposure associated with the census, noting that its existing
internal procedures are sufficient. Nevertheless, we do not believe the
information OMB currently reports to Congress is sufficiently timely or
detailed to provide the level of transparency needed for effective
congressional oversight and cost control, and thus stand by our
recommendation.
Background:
Thorough and comprehensive planning and preparation are crucial to the
ultimate cost-effectiveness of any large, long-term project,
particularly one with the scope, magnitude, and immutable deadlines of
the decennial census. Indeed, the Bureau's past experience has shown
that the lack of proper planning can increase the costs and risks of
downstream operations. Moreover, sound planning is critical to
obtaining congressional support and funding because it helps
demonstrate that the Bureau has chosen the optimal design given various
trade-offs and constraints and that it will effectively manage
operations and control costs.
However, Congress, GAO, the Department of Commerce Inspector General,
and even the Bureau itself have noted how the 2000 Census was marked by
poor planning, which unnecessarily added to the cost, risk, and
controversy of the last national head count. For example, our earlier
work, and that of the Department of Commerce Inspector General,
reported that in planning the 2000 Census, the Bureau, among other
shortcomings,
* did not involve key operations staff in the initial design phases;
* did not translate key performance goals into operational, measurable
terms that could be used as a basis for planning;
* did not develop and document a design until mid-decade; and:
* initially failed to provide sufficient data to stakeholders on the
likely effects of its initiatives for addressing the key goals for the
census--reduced costs and improved accuracy and equity.
Planning weaknesses were not limited to the 2000 Census. In fact, a
variety of problems plagued the planning of the 1990 Census.
To help prevent the Bureau from repeating those mistakes as it plans
the 2010 Census, in our October 2002 report, we recommended that the
Secretary of Commerce direct the Bureau to provide comprehensive
information backed by supporting documentation in its future funding
requests for planning and development activities, including, but not
limited to,
* specific performance goals for the 2010 Census and information on how
the Bureau's programs would contribute to those goals;
* detailed information on program feasibility, priorities, and
potential risks;
* key implementation issues and decision milestones; and:
* performance measures.[Footnote 3]
The consequences of a poorly planned census are high given the billions
of dollars spent on the enterprise and the importance of collecting
quality data. The Constitution requires a census as a basis for
apportioning seats in the House of Representatives. Census data are
also used to redraw congressional districts, allocate billions of
dollars in federal assistance to state and local governments, and
provide information for many other public and private sector purposes.
Scope and Methodology:
As agreed with your offices, our objectives for this report were to
review the Bureau's current plans for the 2010 Census and the extent to
which they might address shortcomings with the 2000 Census, analyze the
Bureau's cost estimates, and assess the rigor of the Bureau's 2010
planning process.
To achieve these three objectives, we interviewed officials from the
Bureau's Decennial Management Division and other units involved with
planning the 2010 Census. We also reviewed relevant design and budget
documents as well as our prior work and that of the Department of
Commerce Inspector General, on planning the 2000 and earlier
censuses.[Footnote 4] We also reviewed reports by the National Academy
of Sciences on planning the 2010 Census.[Footnote 5] We did not
independently verify the cost information the Bureau provided. To help
determine the key elements for successful project planning, we reviewed
a number of guides to project management and business process
reengineering.[Footnote 6]
Our work was performed from January through September 2003 in
accordance with generally accepted government auditing standards. We
requested comments on a draft of this report from the Secretary of
Commerce and the Director of the Office of Management and Budget. On
November 6, 2003, we received the Secretary's written comments on the
draft (see app. I). On October 14, 2003, the Director of OMB forwarded
OMB's comments on the draft (see app. II). They are addressed in the
Agency Comments and Our Evaluation section of this report.
2010 Design Has Potential, but Introduces New Risks:
In designing the 2010 Census, Bureau officials had four principal goals
in mind: (1) increase the relevance and timeliness of census long-form
data, (2) reduce operational risk, (3) increase the coverage and
accuracy of the census, and (4) contain costs. The goals were a direct
response to problems the Bureau experienced in conducting the 2000
Census, such as untimely long-form data; inaccurate maps; coverage
difficulties; and expensive, labor-intensive, and paper-laden field
data collection.
Census Reforms Have Three Key Components:
The Bureau recognized that its traditional approach to counting the
population was insufficient for meeting these four objectives. In its
place, the Bureau developed what it believes is a paradigm shift to
taking the census, basing its reform efforts on three interrelated
components the Bureau refers to as a "three-legged stool.":
Modernizing and Enhancing the MAF/TIGER Address File and Geographic
Database:
The mainstay of a successful census is an accurate address list and its
associated maps. The Bureau uses MAF and TIGER to provide (1) maps for
field operations and data reference, (2) the geographic location of
every structure, (3) address lists for the decennial census, and (4)
names and codes of entities for data tabulation and data for use by the
commercial geographic information systems industry.
The Bureau's experience in conducting the 2000 Census highlighted the
need to update and modernize MAF/TIGER prior to 2010. For example, the
centerlines of streets in TIGER did not accurately reflect their true
geographic locations, which could cause houses to be placed in the
wrong census blocks. Also, according to the Bureau, the 1980s software
used to develop TIGER is now outdated and cumbersome to update. To fix
these and other problems, the Bureau launched the MAF/TIGER
Enhancements Program (MTEP) as part of the 2010 Census modernization
efforts. Its objectives include:
* correcting the locations of each MAF address, street, and other map
features;
* developing and deploying a new MAF/TIGER processing environment;
* expanding and encouraging geographic intergovernmental partnership
programs;
* implementing the Community Address Updating System (CAUS), an
initiative to partner with local governments to update MAF data; and:
* initiating quality assurance evaluation activities.
The Bureau estimates the total cost for these five objectives to be
$536 million. According to Bureau officials, while some elements of MAF
will be improved as part of the overall MAF/TIGER enhancements program,
the primary focus of the effort is on TIGER modernization and data
correction. This modernization program will not reengineer the MAF
process.
Implementing ACS to Collect Long-Form Data on an Ongoing, Annual Basis:
ACS is intended to be a monthly survey of 250,000 households that would
replace the long form used in past decennial censuses. According to the
Bureau, the benefits of ACS include (1) more timely long-form data at
detailed geographic levels that would be as accurate as subnational
annual data from existing surveys, such as the Current Population and
American Housing Surveys, and (2) the ability to improve the accuracy
of the decennial census population counts by eliminating the long-form
questionnaire.
The ACS data will be published annually for geographic areas with
populations over 65,000; as a 3-year average for geographic areas with
populations of 20,000 to 65,000; and as a 5-year average for geographic
areas with populations under 20,000. According to the Bureau, because
of the larger sample size associated with long-form data, the annual
and 3-year average data will be significantly less accurate than the
long-form data. The 5-year data would be about as accurate as the long
form.
Developing, Testing, and Implementing a Short-Form-Only Census; Taking
Advantage of the Benefits Gained by the First Two Efforts:
The Bureau believes that eliminating the long form will result in a
number of benefits to decennial data collection and general field
operations. For example, according to the Bureau, the reduction in
paper will enable it to process data with three data capture centers
instead of the four centers used during Census 2000. The Bureau also
would not need as many local census offices, thereby allowing it to
reduce the rolls of clerical and administrative staff.
According to the Bureau, a short-form-only census also allows the
Bureau to use such technology as handheld mobile computing devices so
that enumerators can locate and update data on housing units, help
conduct interviews, transmit data directly to the data capture centers,
and receive regularly updated field assignments. The devices will be
linked to the satellite-based Global Positioning System (GPS) to enable
field workers to locate addresses more precisely and efficiently. The
Bureau also plans to incorporate changes that are not dependent on ACS
and MAF/TIGER. For example, the Bureau plans to expand the respondents'
ability to complete their questionnaires via the Internet.
2010 Design Is Linked to Goals, but Challenges Remain:
As shown in table 1, the Bureau's three-legged stool strategy is
generally aligned with three of its four key goals for the 2010 Census
and, if successful, could put the Bureau on track toward achieving
them.
Table 1: The Three-Legged Stool Strategy Could Help the Bureau Progress
toward Its Key Goals for the 2010 Census:
Goal: Increase the relevance and timeliness of census long-form data;
Strategy for achieving goal: Implement ACS; Rationale: ACS, if
implemented nationwide, is to replace the long form and provide annual
data for areas with populations over 65,000 and multiyear averages for
smaller geographic areas. This compares to the long form's once-every-
10-year data.
Goal: Reduce operational risk; Strategy for achieving goal: Early
planning and testing, short-form-only; census questionnaire;
Rationale: The Bureau believes that testing a design early in the
decade will identify design flaws and allow the Bureau to correct them
early in the process. Also, a short-form-only census should be easier
to implement since Bureau staff will only have to plan systems for one
type of form and handle much less paper.
Goal: Increase coverage and accuracy; Strategy for achieving goal:
Short-form-only census, MAF/TIGER enhancements; Rationale: The Bureau
believes that a short-form-only census will increase the mail response
rate by 8 percentage points because, historically, more people have
responded to the short form than the long form, and the Bureau has
found that the quality of the data is better when supplied directly by
respondents. The Bureau also plans to do a targeted second mailing to
nonresponders that should also boost the response rate. These efforts
combined with other planned improvements, such as the MAF/TIGER
enhancements, would improve overall accuracy and coverage.
Goal: Contain costs; Strategy for achieving goal: Short-form-only
census; Rationale: The higher response rates produced by the short form
will reduce the Bureau's nonresponse follow-up workload, one of the
largest drivers of census costs. The Bureau also believes that a short-
form-only census will result in less paper, and thus enable the Bureau
to get by with three rather than four data capture centers (as was the
case in 2000) and fewer local census offices.
Source: GAO analysis of U.S. Census Bureau data.
[End of table]
Less clear is how the Bureau will achieve its goal of reducing
operational risk using its current plan. Although the Bureau's position
that early testing will enable it to identify and correct flaws is both
a common sense business practice and supported by its past experience
(assuming its testing program is adequately designed), as described
below, the operational and other hurdles associated with successfully
implementing the three-legged stool actually introduce new risks and
challenges. This does not necessarily mean that the Bureau's design is
flawed. To the contrary, the obstacles to a cost-effective head count
call for the Bureau to consider bold and innovative initiatives, and
these are not risk free.
At the same time, given the enormity of the census and all of its
complexities, the three-legged stool by itself will not automatically
guarantee the successful accomplishment of the Bureau's goals. Our work
on transforming agencies into high-performing organizations has
underscored the importance of an agency's leadership and core business
practices. Critical success factors include, among others, effective
communication strategies to ensure coordination, synergy, and
integration; strategic planning; aligning the agency's organization to
be consistent with the goals and objectives established in the
strategic plan; and effective performance, financial, acquisition, and
information technology management. In all, the Bureau faces at least
three key challenges.
Challenge #1: Securing Congressional Approval:
Among the more significant challenges the Bureau faces is securing
congressional approval for its proposed approach. As we noted in our
January 2003 performance and accountability report, congressional
support for the 2010 design is necessary to ensure adequate planning,
testing, and funding levels.[Footnote 7] Conversely, the lack of an
agreed-upon design raises the risk that basic design elements might
change in the years ahead, while the opportunities to test those
changes and integrate them with other operations will diminish. In
other words, in order for the Bureau to conduct proper planning and
development activities, the basic design of the 2010 Census needs to be
stable.
According to the Bureau, a go/no-go decision on key aspects of the
design--a short-form-only census and replacing the long form with ACS-
-will need to be made around 2006. Bureau officials told us that if ACS
were dropped after 2007, the Bureau would not be able to reinstate the
long form with the short form in 2010 because of logistical obstacles.
They noted that the Bureau is already testing the short-form-only
census and, in late-2005 or early-2006, expects to sign a contract for
data capture operations. If the Bureau had to revert to a long-form
census at that point, it would add significant risks and costs to the
2010 Census.
During the 2000 Census, the lack of an agreement between the
administration and Congress on the fundamental design--and
particularly, the Bureau's planned use of sampling--increased the
likelihood of an unsuccessful head count and was one of the principal
reasons why, in 1997, we designated the 2000 Census a high-risk area.
Members of Congress questioned the use of sampling and estimation for
legal and methodological reasons. Contributing to Congress's skepticism
was the Bureau's failure to provide sufficiently detailed data on the
effects of its proposed approach. Although the U.S. Supreme Court
settled the dispute in January 1999,[Footnote 8] the Bureau ultimately
wound up having to plan for both a "traditional" census and one
involving sampling, which added to the costs and risks of the 2000
decennial census.
To help secure congressional support for its 2010 reform efforts, it
will be important for the Bureau to convincingly demonstrate that it
has chosen the optimum design given various resource and other
constraints and that it will effectively manage operations and costs. A
critical first step in this regard is to have comprehensive and
transparent information that lays out the specifics of the Bureau's
plans, explains their benefits, and supports assumptions. However, as
discussed more fully in the next section, while the Bureau's planning
for the 2010 Census has improved compared with its efforts for the 2000
Census, certain information gaps remain. For example, the Bureau's most
recent budget submissions have not included complete life cycle cost
estimates that could enable Congress to make more informed decisions
about the cost implications of the three-legged stool design, including
ACS.
Challenge #2: Resolving Uncertainties That Surround ACS:
Most of the reforms, savings, and improvements in accuracy the Bureau
anticipates will not be possible unless it conducts a short-form-only
census. However, the Bureau's planned replacement for the long form,
ACS, faces methodological and other questions that need to be resolved
soon.[Footnote 9] Consequently, the Bureau is taking a significant risk
by pinning the success of its reform efforts largely on a survey that
may not be an adequate replacement for the long form. The Bureau
believes that without ACS, it will need to repeat the Census 2000
design.
One methodological question is whether to administer ACS as a mandatory
or voluntary survey. Under the Bureau's current approach, survey
recipients will be legally required to respond to ACS. However, in
response to congressional concerns that a mandatory survey is
intrusive, the Bureau tested conducting ACS as a voluntary survey.
Based on the results of the test, the Bureau estimates that a voluntary
survey could produce a response rate around 4.2 percentage points lower
than a mandatory survey. The Bureau estimates that costs would increase
by $59.2 million per year to maintain the same level of reliability
achieved from a mandatory survey.
Moreover, the Bureau's efforts to ensure that ACS data will serve as a
satisfactory replacement for the long-form data are not yet complete.
Among the remaining issues, most of which are critical to the
reliability of the small geographic area ACS data, are the following:
1. Benchmarking ACS data or small geographic areas to the population
counts and characteristics from the 2010 short form.
2. Inconsistency of ACS residency rules--which determine the geographic
area in which a person is supposed to be counted--with those used for
the census.
3. Consistency of ACS data with long-form data from the 2000 Census.
4. For multiyear averages of ACS data for small areas, consistency with
annual ACS data for larger areas and utilization as a measure of
change.
Challenge #3: Successfully Implementing Each Leg of the Bureau's
Design:
Each leg of the Bureau's three-legged stool is dependent on the other;
that is, the implementation of one leg allows the other two legs to
operate successfully. For example, ACS is facilitated by first updating
the MAF/TIGER database for the ACS sample. Similarly, as noted above,
the Bureau's plan to conduct a short-form-only enumeration depends on
ACS. Consequently, the Bureau's design assumes that by 2008, (1) ACS
will be in place nationwide and producing data, (2) a GPS-aligned and
modernized MAF/TIGER will be available, and (3) all reengineering
efforts will be complete to allow for a true dress rehearsal.
Completing any one of these tasks would be a considerable undertaking;
for 2010, the Bureau plans to develop, refine, and integrate all three
in the space of just a few years.
Moreover, the Bureau has no contingency plans other than to revert to a
"traditional" census. According to the Bureau, while the failure of any
one leg would not doom the census, it could jeopardize the Bureau's
goals. For example, if the MAF/TIGER modernization is not completed on
schedule, the Bureau would be unable to employ the GPS-enabled handheld
mobile computing devices that enumerators are to use when conducting
nonresponse follow-up. This in turn could affect the efficiency of the
effort and the quality of the data collected. In addition, the Bureau
would not have time to conduct the research and testing necessary to
improve the long form based on lessons learned in the 2000 Census.
The Bureau's Plans for Measuring Accuracy and Coverage in 2010:
Because of limitations in census taking methods, some degree of error
in the form of persons missed or counted more than once is inevitable.
Since 1980, the Bureau has used statistical methods to generate
detailed measures of the undercounts of particular ethnic, racial, and
other population groups. To assess the quality of population data for
the 2000 Census and to possibly adjust for any errors, the Bureau
conducted the Accuracy and Coverage Evaluation (A.C.E.) program.
Although the U.S. Supreme Court ruled in 1999 that the Census Act
prohibited the use of statistical sampling for purposes of apportioning
seats in the House of Representatives, the Court did not specifically
address the use of statistical sampling for other purposes.[Footnote
10]
In March 2001, the Acting Director of the U.S. Census Bureau
recommended to the Secretary of Commerce that only unadjusted data be
released for purposes of congressional redistricting. The Acting
Director made this recommendation when, after considerable research,
the Bureau was unable to conclude that the adjusted data were more
accurate for use in redistricting. Specifically, the Acting Director
cited the apparent inconsistency in population growth over the decade
as estimated by the A.C.E., and demographic analysis, which estimated
the population using birth, death, and similar records. He noted that
the inconsistency raised the possibility of an unidentified error in
either the A.C.E. estimates or the census numbers, and the
inconsistency could not be resolved prior to April 1, 2001, the legally
mandated deadline for releasing redistricting data. Later that year,
following additional research, the Acting Director decided against
using adjusted census data for nonredistricting purposes, such as
allocating federal aid, because A.C.E. estimates failed to identify a
significant number of people erroneously included in the census. The
Acting Director noted that "this finding of substantial error, in
conjunction with remaining uncertainties, necessitates that revisions,
based on additional review and analysis, be made to the A.C.E.
estimates before any potential uses of these data can be considered.":
As the Bureau turned toward the 2010 Census, it needed to decide
whether it would have a coverage measurement program and how the
results would be used. Because of the 1999 U.S. Supreme Court ruling
noted earlier, the Bureau could not use coverage measurement results to
adjust census data for purposes of congressional apportionment.
However, adjusting census data for other purposes remained an open
question.
In our January 2003 report on the objectives and results of the 2000
A.C.E. program, we noted that an evaluation of the accuracy and
completeness of the census is critical given the many uses of census
data, the importance of identifying the magnitude and characteristics
of any undercounts and overcounts, and the cost of the census
overall.[Footnote 11] We cautioned that the longer the 2010 planning
process continues without a firm decision on the role of coverage
measurement, the greater the risk of wasted resources and disappointing
results.
Consequently, we recommended that the Bureau, in conjunction with
Congress and other stakeholders, come to an early decision on whether
and how coverage measurement will be used in the 2010 Census. In
reaching this decision, we recommended that the Bureau (1) demonstrate
both the operational and technical feasibility of its coverage
measurement methods, (2) determine the level of geography at which
coverage can be reliably measured, (3) keep Congress and other
stakeholders informed of its plans, and (4) adequately test its
coverage measurement methodology prior to full implementation.
The Bureau agreed with our recommendations, noting that we had
identified important steps that should be followed in developing a
coverage measurement methodology for the 2010 Census. While certain
aspects of the Bureau's coverage measurement plans are still being
developed, the Bureau is not currently planning to develop a procedure
that would allow it to adjust census numbers for purposes of
redistricting.
According to the Director of the U.S. Census Bureau, although the
Bureau plans to evaluate the accuracy of the coverage it achieves in
the 2010 Census, its experience during the 2000 Census demonstrated
"that the science is insufficiently advanced to allow making
statistical adjustment to population counts of a successful decennial
census in which the percentage of error is presumed to be so small that
adjustment would introduce as much or more error than it was designed
to correct.":
Furthermore, irrespective of whether it is both legal and appropriate
to do so, the Bureau does not believe that it can both collect coverage
measurement data and complete the analysis of those data's accuracy in
time to deliver the information to the states to meet their
redistricting deadlines.
Although the Bureau's experience during the 2000 Census shows that its
approach to measuring coverage needs to be improved if it is to be used
to adjust census numbers, the Bureau has not yet determined the
feasibility of refinements to the 2000 approach or alternative
methodologies. Consequently, the Bureau's decision on how coverage
evaluation data will be used in 2010 appears to be premature. Indeed,
while the Bureau has reported that the 2000 Census had better coverage
compared to the 1990 Census, as noted below, the U.S. population is
becoming increasingly difficult to count, a factor that could affect
the quality of the 2010 Census.
More generally, the decennial census is an inherently fragile endeavor,
where an accurate population count requires the near-perfect alignment
of a myriad of factors ranging from the successful execution of dozens
of census-taking operations to the public's willingness to cooperate
with enumerators. External factors such as the state of the economy and
world events might also affect the outcome of the census. The bottom
line is that while the census is under way, the tolerance for any
breakdowns is quite small.
Therefore, the Bureau's ability to maintain the level of quality
reported for the 2000 Census is far from guaranteed. Thus, to ensure
that the nation uses the best available census data, it will be
important for the Bureau to research procedures that depending on what
the results of the coverage evaluation say about the quality of the
census data, would allow adjustment, if necessary, for those purposes
for which it is both legal and appropriate to do so. The Bureau should
conduct this effort on a timetable that allows it to adequately test
and refine those procedures, as well as obtain input from both majority
and minority parties in the Senate and House of Representatives.
Escalating Census Costs Call for Transparent Budget Data and Better
Reporting:
In June 2001, the Bureau estimated that the reengineered 2010 Census
would cost $11.3 billion in current dollars. This would make the 2010
head count the most expensive in the nation's history, even after
adjusting for inflation. According to the Bureau estimates in June
2001, a repeat of the 2000 approach would cost even more, over $11.7
billion. This estimate of repeating the 2000 approach was revised to
approximately $12.2 billion in April 2003. Moreover, the actual cost of
the census could end up considerably higher as the Bureau's initial
cost projections for previous censuses proved to be too low because of
such factors as unforeseen operational problems or changes to the
fundamental design. For example, while the Bureau estimated that the
2000 Census would cost around $4 billion using sampling, and that a
traditional census without sampling would cost around $5 billion, the
final price tag for the 2000 Census (without sampling) was over $6.5
billion.
Census Costs Continue to Grow:
The Bureau's cost projections for the 2010 decennial census continue an
escalating trend. As shown in figure 1, in constant 2000 dollars, the
estimated $9.3 billion cost of the 2010 Census represents a tenfold
jump over the $920 million spent on the 1970 Census (as noted above,
the Bureau estimates the 2010 Census will cost $11.3 billion in current
dollars).
Figure 1: Decennial Census Costs from 1970 through 2010 (Projected) in
Constant 2000 Dollars:
[See PDF for image]
[End of figure]
Although some cost growth can be expected in part because the number of
housing units--and hence the Bureau's workload--has gotten larger, the
cost growth has far exceeded the housing unit increase. The Bureau
estimates that the number of housing units for the 2010 Census will
increase by 10 percent over 2000 Census levels. Meanwhile, the average
cost per housing unit for 2010 is expected to increase by approximately
29 percent from 2000 levels (from $56 to $72), nearly five and a half
times greater than the $13 it cost to count each household in 1970 (see
fig. 2).[Footnote 12]
Figure 2: Decennial Census Average Cost per Housing Unit (in Constant
Fiscal Year 2000 Dollars):
[See PDF for image]
[End of figure]
As for previous censuses, the major cost for the 2010 Census is what
the Bureau calls "field data collection and support systems." Over half
of decennial census life cycle costs are attributed to this area.
Specific components include the costly and labor-intensive nonresponse
follow-up operation as well as support activities such as the opening
and staffing of hundreds of temporary local census offices.
One reason why field data collection is so expensive is because the
Bureau is finding it increasingly difficult to locate people and get
them to participate in the census. According to Bureau officials,
societal trends, such as the increasing number of respondents who do
not speak English, the growing difficulty of finding respondents at
home, and the general increase of privacy concerns, impede a cost-
effective head count. Further, the legal requirement to count everyone
leads the Bureau to employ operations that only marginally improve
coverage but that are relatively expensive to conduct.
Societal changes have also reduced the cost-effectiveness of the
census, and it has become increasingly difficult to simply stay on par
with the results of previous enumerations. For example, during the 1990
Census, the Bureau spent $0.88 per housing unit (in 2000 dollars) to
market the census and encourage participation and achieved a response
rate of 65 percent. During the 2000 Census, the Bureau spent $3.19 per
housing unit (in 2000 dollars) to promote participation, but the
response rate was 64 percent.
Cost Estimates Need Greater Transparency:
The constitutional mandate to count the nation's population explicitly
commits or "exposes" the government to spending money on the census
each decade. In this way, the census is similar to other fiscal
exposures such as retirement benefits, environmental cleanup costs, and
the payment of Social Security benefits in that the government is
obligated to a certain level of future outlays.
Early in each census cycle, expenditures are relatively low as the
Bureau plans the census and conducts various tests. As the decade
continues, spending increases, spiking during the decennial year when
costly data collection activities take place. As shown in figure 3,
during the 2000 Census, $4.1 billion--almost two-thirds of the money
spent on the entire census--was spent in fiscal year 2000 alone.
Figure 3: 2000 Census Life Cycle Costs by Fiscal Year:
[See PDF for image]
[End of figure]
Current budget reporting, however, does not always fully capture or
require explicit consideration of some future fiscal exposures. In
fact, this is particularly true with the census, as annual budget
requests and reports provided to Congress early in the decennial census
life cycle do not reflect the full cash consequences of the spending
and policy decisions. Thus, as it begins funding the 2010 Census early
in the decade at relatively low levels, Congress will have implicitly
accepted a future spike in costs--essentially a balloon payment in
2010--without requiring the Bureau to clearly define what those costs
might be, why they are justified, and what alternatives might exist.
As we noted in our January 2003 report on improving the budgetary focus
on long-term costs and uncertainties, information on the existence and
estimated cost of fiscal exposures needs to be considered along with
other factors when making policy decisions.[Footnote 13] With respect
to the census, not capturing the long-term costs of annual spending
decisions limits Congress's ability to control the government's
exposure at the time decisions are made, consider trade-offs with other
national priorities, and curtail the growth in census costs.
Consequently, fiscal transparency is critical to better reflect the
magnitude of the government's long-term spending on the census and
signal unanticipated cost growth. Greater fiscal transparency can also
facilitate an independent review and provide an opportunity to improve
stakeholder confidence and commitment to the Bureau's reengineered
decennial census design.
Our January 2003 report noted that increased supplemental reporting
could help improve fiscal transparency and described several options
for how to accomplish this. Although that report recommended that OMB
consider implementing these options governmentwide, at least two
options could be adapted specifically for the Bureau and its parent
agency, the Department of Commerce. The two options are (1) annually
reporting the planned life cycle cash flow and explaining any material
changes from previous plans (currently, the Bureau does not make this
information available) and (2) setting triggers to signal when the
amount of money expected to be spent in any one year exceeded a
predetermined amount. Combined, these actions could prompt more
explicit deliberations on the cost and affordability of the census and
help inform specific cost control measures by Congress, if warranted.
More Information Needed on Underlying Budget Assumptions:
The assumptions the Bureau used to develop the life cycle cost estimate
could also benefit from greater transparency.[Footnote 14] More robust
information on the likelihood that the values the Bureau assigned to
key cost drivers might differ from those initially assumed, and the
impact that any differences would have on the total life cycle cost,
could provide Congress with better information on the range and
probability of the fiscal exposure the nation faces from the upcoming
census.
As shown in figure 4, the Bureau derived the baseline for its 2010 cost
estimate using the actual cost of the 2000 Census and assumptions about
certain cost drivers, estimating the cost of the 2010 Census as if the
Bureau were to repeat its 2000 design. The key assumptions include:
* a 35 percent decrease in enumerator productivity,
* a pay rate increase for census workers from 2000 levels,
* a mail-back response rate lower than Census 2000 levels,[Footnote 15]
and:
* inflation.
Figure 4: Bureau's June 2001 Estimating Methodology for Census 2010
Life Cycle Costs:
[See PDF for image]
[End of figure]
The projected costs and savings of repeating the 2000 design versus the
Bureau's approach based on the three-legged stool, are shown in table
2.
Table 2: Projected Costs from June 2001 Estimate of Reengineering the
2010 Census:
Dollars in millions.
Budget frameworks: Program Development and Management;
Repeated 2000 Census design (baseline): 183;
Reengineered 2010 Census (three-legged stool): 247;
Savings (increases): (64).
Budget frameworks: Data Content and Products;
Repeated 2000 Census design (baseline): 765;
Reengineered 2010 Census (three-legged stool): 705;
Savings (increases): 60.
Budget frameworks: Field Data Collection and Systems;
Repeated 2000 Census design (baseline): 7,880;
Reengineered 2010 Census (three-legged stool): 5,640;
Savings (increases): 2,240.
Budget frameworks: Address List Compilation;
Repeated 2000 Census design (baseline): 610;
Reengineered 2010 Census (three-legged stool): 455;
Savings (increases): 155.
Budget frameworks: Automated Data Processing;
Repeated 2000 Census design (baseline): 1,265;
Reengineered 2010 Census (three-legged stool): 840;
Savings (increases): 425.
Budget frameworks: Testing, Evaluation, and Rehearsal;
Repeated 2000 Census design (baseline): 391;
Reengineered 2010 Census (three-legged stool): 467;
Savings (increases): (76).
Budget frameworks: Island Areas;
Repeated 2000 Census design (baseline): 163;
Reengineered 2010 Census (three-legged stool): 153;
Savings (increases): 10.
Budget frameworks: Marketing and Communications;
Repeated 2000 Census design (baseline): 468;
Reengineered 2010 Census (three-legged stool): 518;
Savings (increases): (50).
Budget frameworks: ACS;
Repeated 2000 Census design (baseline): [Empty];
Reengineered 2010 Census (three-legged stool): 1,720;
Savings (increases): (1,720).
Budget frameworks: MAF/TIGER;
Repeated 2000 Census design (baseline): [Empty];
Reengineered 2010 Census (three-legged stool): 535;
Savings (increases): (535).
Total;
Repeated 2000 Census design (baseline): 11,725;
Reengineered 2010 Census (three-legged stool): 11,280[A];
Savings (increases): 445.
Source: U.S. Census Bureau.
Note: ACS and MAF/TIGER modernization spending is not included under
the scenario of repeating the 2000 Census design for the 2010 Census.
[A] The Bureau revised its estimates in April 2003, but we did not have
the breakdown by budget framework to revise this table.
[End of table]
Transparent information is especially important since decennial cost
estimates are sensitive to many key assumptions. In fact, for the 2000
Census, the Bureau's supplemental funding request for $1.7 billion in
fiscal year 2000 primarily involved changes in assumptions related to
increased workload, reduced employee productivity, and increased
advertising.
Given the cost estimates' sensitivity to key assumptions, greater
transparency could be obtained by showing the range and likelihood of
how true cost drivers could differ from those assumed and how those
differences would affect estimates of total cost. Thus, if early
research and testing show that response rates may be higher than
originally anticipated, or that enumerator productivity could be better
than expected, the Bureau can report on the nature of its changing
assumptions and its effect on life cycle costs. Also important, by
providing information on the likely accuracy of assumptions concerning
cost drivers, the Bureau would better enable Congress to consider
funding levels in an uncertain environment. Other key areas in which
changes in assumptions can greatly affect costs are salary rates for
enumerators, the future price of handheld mobile computing devices, and
inflation.
Our prior work has described how agencies provide supporting
information when developing budget assumptions. For example, the
Nuclear Regulatory Commission identifies a basis and a certainty level
for its budget assumptions used for internal reporting. A basis
summarizes the facts that were evaluated to justify the assumption,
while a certainty level depicts the likelihood of its occurrence as
high, medium, or low.[Footnote 16]
Finally, it is important to have timely cost information for
congressional decision making. The Bureau's life cycle estimates were
updated in April 2003 after being first presented in June 2001--nearly
a 2-year interval. In addition, when we requested additional
information on life cycle costs the Bureau took several months to
provide information on its life cycle cost estimates and assumptions,
ultimately revising its total cost estimates before providing us with
the data.
2010 Planning Process Is Better Than Past Efforts, but Rigor Is Uneven:
The Bureau has taken several positive steps to correct problems
encountered planning past censuses, and the Bureau appears to be
further along in planning the 2010 Census than at this same point
during the 2000 Census cycle. Although an improvement over past
efforts, the Bureau's 2010 planning process still contains certain weak
points that if not addressed could undermine a cost-effective head
count and make it more difficult to obtain the support of Congress and
other stakeholders.
The Bureau Improved Its Planning Process, but Key Elements Are Still
Lacking:
The characteristics of the census--long-term, large-scale, high-risk,
costly, and politically sensitive--together make a cost-effective
enumeration a monumental project management challenge, one that demands
meticulous planning. To help determine the principal ingredients of
successful project planning, we reviewed a number of guides to project
management and business process reengineering. Although there is no one
best approach to project planning, the guides we reviewed contained
many elements in common, including the following:
* Developing a project plan. The plan should consider all phases of the
project and should have clear and measurable goals; all assumptions,
schedules, and deadlines clearly stated; and needed skills and
resources identified.
* Evaluating human resource implications. This includes assessing
needed competencies and how they will be acquired and retained.
* Involving stakeholders and incorporating lessons learned.
Stakeholders--both internal and external to an organization--have
skills and knowledge that could contribute to a project and should be
involved in the decision-making process. An organization should focus
on the highest priority stakeholder needs and mission goals. Evaluating
past performance and capitalizing on lessons learned is also important
for improving performance.
* Analyzing and mitigating risks. This involves identifying, analyzing,
prioritizing, and documenting risks. Ideally, more than one alternative
should be assessed.
* Monitoring progress. Measurable performance goals should be
identified and performance data should be gathered to determine how
well the goals are being achieved.
The Bureau has made considerable progress in planning the 2010 Census,
and some of the positive steps taken to date include the following
efforts.
* Early in the decade, senior Bureau staff considered various goals for
the 2010 Census and articulated a design strategy to achieve those
goals. Senior Bureau officials collaborated on this initial design plan
to set the stage for further refinements during later field testing and
research activities.[Footnote 17]
* The Bureau has involved experienced staff in the design process
through cross-divisional planning groups. Staff involved in these
planning groups will ultimately be responsible for implementing the
2010 Census. According to Bureau officials, this is a departure from
the 2000 Census planning effort when Bureau staff with little
experience in conducting the census played a key role in designing the
decennial census, which resulted in impractical reforms that could not
be implemented.
* The Bureau has recognized the importance of strategically managing
its human capital to meet future requirements. The planning and
development of the 2010 Census will take place at a time when the
Bureau could find itself experiencing substantial employee turnover
(three senior Bureau managers left the agency in 2002, and according to
a report by the Department of Commerce Inspector General, the Bureau
could lose through retirement around half of the senior staff who
carried out the 2000 Census).[Footnote 18] The Bureau, as part of a
broader risk assessment, plans to provide less experienced staff the
opportunity to obtain operational experience prior to the actual 2010
Census. In addition, the Bureau has provided training in project
management and has encouraged staff to take training courses in
management and planning.
However, other aspects of the Bureau's 2010 planning process could be
improved. A more rigorous plan would better position the Bureau to
fulfill its key objectives for the 2010 Census and help demonstrate to
Congress and other stakeholders that it can effectively design and
manage operations as well as control costs.
Developing a Project Plan:
Although the Bureau has developed project plans for some of the key
components of its 2010 strategy, the Bureau has not yet crafted an
overall project plan that (1) includes milestones for completing key
activities; (2) itemizes the estimated cost of each component; (3)
articulates a clear system of coordination among project components;
and (4) translates key goals into measurable, operational terms to
provide meaningful guidance for planning and measuring progress. OMB
Circular A-11[Footnote 19] specifies that an agency's general goals
should be sufficiently precise to direct and guide agency staff in
actions that carry out the agency's mission and aid the agency in
developing annual performance goals.
The importance of this information for improving accountability and
performance can be seen, for example, in the Bureau's principal goal to
increase coverage and accuracy. Though laudable, the Bureau has yet to
assign any numbers to this goal. This makes it difficult to evaluate
the costs and benefits of alternative designs, determine the level of
resources needed to achieve this goal, measure the Bureau's progress,
or hold managers accountable for results.
Bureau managers provided us with several documents that pieced together
present 2010 Census goals and strategies, life cycle costs, and
schedules, but no single, comprehensive document exists that integrates
this information. For example, the Bureau's life cycle cost estimates
and information on its performance goals were contained in two separate
documents, making it hard to see the connection between cost and the
Bureau's objectives. Likewise, a draft document, entitled 2010
Reengineered Census Milestone Schedule, included various milestones by
fiscal quarter, but did not contain information on dependencies and
interrelationships among the various aspects of the project.
In its 2001 letter to the Bureau's acting director, the National
Academy of Sciences' (NAS) Panel on Research on Future Census Methods
raised similar concerns about the need for a coherent project plan. The
panel noted that it wanted "to see a clearer case for components of the
2010 census strategy, itemizing the goals, costs, and benefits of each
initiative and indicating how they integrate and contribute to a high
quality census." To that end, NAS recommended that the Bureau develop
what it called a business plan for 2010.
Incorporating Lessons Learned:
The Bureau is making an effort to develop and incorporate the lessons
learned from the 2000 Census and, in fact, created an elaborate
evaluation program to help inform this effort. Moreover, the Bureau
chartered 11 planning groups consisting of knowledgeable census staff
(see app. III for the 2010 planning organization). However, the
Bureau's ability to build on the results of 2000 could be hampered by
the fact that while the evaluation program assessed numerous aspects of
the census, the Bureau still lacks data and information on the
performance of key census activities, as well as on how specific census
operations contributed to two of the Bureau's key goals for 2000:
improved accuracy and cost-effectiveness.
For example, as noted earlier, the cost of the 2010 Census is
increasing relative to 2000 partly because the Bureau expects
nonresponse follow-up enumerators will be less productive in 2010.
Because of various societal factors, it will simply take enumerators
more time to complete their work. And yet, despite the importance of
accurate productivity data to inform the Bureau's planning and
budgeting processes for 2010, the Bureau had trouble obtaining quality
productivity data following the 2000 Census. Although the Bureau later
committed additional resources to refine the numbers, the adjustment
was coarse and addressed just one of the two known problems. Moreover,
because of differences in the way the Bureau measured staffing levels
and hours worked from census to census, none of the productivity data
from the last few censuses are comparable.[Footnote 20]
Another area in which the Bureau lacks useful performance information
is in the extent to which the dozen or so separate activities used to
build MAF in 2000 contributed to its overall accuracy relative to one
another. Without this information, the Bureau has limited data with
which to guide investment and trade-off questions for 2010, such as
which activity provided the biggest "bang for the buck" and should thus
be repeated, or whether it would be more effective for the Bureau to
improve accuracy and coverage by putting more resources into MAF-
building activities or some other operation altogether, such as
marketing.
To date, the Bureau's planning groups have incorporated a variety of
lessons learned from the evaluations of the 2000 Census. As an example,
the Coverage Improvement Planning Group observed an increase in
inconsistent responses from 2000 Census compared to the previous census
(e.g., some questionnaires were marked "uninhabited," but individuals
were enumerated at the sites). According to a Bureau official, one
hypothesis for the higher rate of inconsistent responses was that
enumerators were encouraged to fill in information even when not all of
the relevant information was known. The Bureau plans to address this
issue by building in "edits" to its planned handheld mobile computing
devices so that inconsistent data cannot be entered.[Footnote 21] In
addition, the Coverage Improvement Planning Group also looked at the
2000 Census experience to provide recommendations for the Bureau's 2004
test.
Analyzing and Mitigating Risks:
Risk management is important for preparing for contingencies or changes
in the external operating environment. At the time of our review, the
Bureau had completed a risk assessment of some aspects of its
operations as part of its OMB Circular A-11, Exhibit 300
submission,[Footnote 22] and for certain aspects of the reengineering
efforts.[Footnote 23] However, the Bureau had not developed a risk
assessment that addressed the entire 2010 Census, including ACS and the
MAF/TIGER modernization.
The risk assessment for the reengineering effort uses a consistent
scoring system to assess the severity of the risks identified and
addresses various contingencies and mitigation strategies, such as
preparing for the retirement of key personnel and using succession
planning to offset the attrition. The scoring system and how it was
applied is clearly described in the plan, making it easy to evaluate
the way it was used.
However, the assessment does not provide extensive detail on the
mitigation actions proposed. Also, it does not indicate how risks were
identified and whether any risks were excluded. A notable exclusion was
that it did not address the risks if ACS or MAF/TIGER fail or are not
funded and the impact this might have on the census as a whole.
As mentioned earlier, the Bureau's three-legged stool strategy assumes
that all three legs must work together to achieve its goals. One of the
reasons for doing a risk analysis is to prepare to make trade-offs when
faced with inevitable budgetary pressures, operational delays, or other
risks. Lacking information on trade-offs, the Bureau maintains that its
only alternative to the reengineering is to repeat the 2000 Census
design, an approach that Bureau officials believe will be extremely
expensive.
Conclusions:
The obstacles to conducting a cost-effective census have grown with
each decade, and as the Bureau looks toward 2010, it confronts its
biggest challenge yet. Consequently, the Bureau will need to balance
the growing cost, complexity, and political sensitivity of the census
with meticulous planning. As the Bureau's past experience has shown,
early investments in planning can help reduce the costs and risks of
its downstream operations.
Moreover, a rigorous plan is essential for securing early agreement
between the Bureau and Congress on the Bureau's fundamental strategy
for 2010. Congressional support--regardless of whether the Bureau's
current approach or an alternative is ultimately selected--is crucial
for creating a stable environment in which to prepare for the census
and avoiding a repeat of the 2000 Census when disagreement over the
Bureau's methodology led to late design changes and additional costs
and risks. The Bureau has laid out an ambitious schedule of planning,
testing, and evaluation for the coming years, culminating with a "dress
rehearsal" in 2008. While midcourse corrections are to be expected as a
result of these efforts, it will be important for the Bureau to proceed
with as few alterations to its fundamental strategy as possible so that
all of the operations used in 2010 have been thoroughly road tested.
The Bureau appears to be further along in planning the 2010 Census
compared to a similar point during the 2000 Census cycle, and its
efforts to enhance past planning practices are commendable. Focusing
its activities on early design, research, and testing and organizing
its reengineering activities around cross-divisional planning groups,
are just some of the noteworthy improvements the Bureau has made.
However, the Bureau's plans for 2010, while not unreasonable on the
surface, lack a substantial amount of supporting analysis, budgetary
transparency, and other information, making it difficult for us,
Congress, and other stakeholders to properly assess the feasibility of
the Bureau's design and the extent to which it could lead to greater
cost-effectiveness compared to alternative approaches.
Questions surrounding the Bureau's underlying budget assumptions;
uncertainties over ACS; the failure to translate key goals into
measurable, operational terms; and the lack of important performance
data from the 2000 Census to inform 2010 decision making are just some
of the problematic aspects of the 2010 planning process. More than
simply paperwork or documentation issues, this information is essential
for improving the performance and accountability of the Bureau and of
the decennial census in particular.
To be sure, some challenges are to be expected in an endeavor as
demanding as counting a population that is mobile and demographically
complex and whose members reside under a multitude of living
arrangements. Further, shortcomings with prior censuses call for the
Bureau to consider bold initiatives for 2010 that entail some risk.
However, if Congress is to accept and fund the Bureau's approach--now
estimated at more than $11 billion--then the Bureau needs to more
effectively demonstrate that it has (1) selected a design that will
lead to the most cost-effective results and (2) establish a rigorous
capacity to manage risks, control costs, and deliver a successful head
count. Moreover, to ensure the nation uses the best available data, it
will be important for the Bureau to research procedures that would
allow it to adjust census results for purposes for which it is both
legal and appropriate to do so, if it is determined that the adjusted
figures would provide greater accuracy than the enumeration data. Such
procedures could function as a safety net should there be problems with
the initial census count.
It will also be important for policymakers to consider, early in the
decade, the long-term costs associated with the census and finding the
right balance between controlling mushrooming costs and improving
accuracy. Although initial spending on the census is relatively low, it
will accelerate in the years ahead, culminating with a balloon payment
in 2010 when data collection and other costly operations take place.
Greater fiscal transparency prior to getting locked into a particular
level of spending could help inform deliberations on the extent to
which (1) the cost of the census is reasonable, (2) trade-offs will
need to be made with competing national priorities, and (3) additional
dollars spent on the census yield better quality data.
Just over 6 years remain until Census Day 2010. While this might seem
like an ample amount of time to shore up the Bureau's planning process
and take steps to control costs, past experience has shown that the
chain of interrelated preparations that need to occur at specific times
and in the right sequence leave little room for delay or missteps.
Recommendations for Executive Action:
To help control the cost of the 2010 Census and inform deliberations on
the acceptability of those costs, we recommend that the Director of the
Office of Management and Budget take steps to ensure that the Bureau
improves the transparency of the fiscal exposure associated with the
census. Specifically, OMB should ensure that the Bureau, in a
notational item in the Program and Financing schedule of the
President's budget, include an updated estimate of the life cycle costs
of the census and the amount of money the Bureau expects to spend in
each year of the cycle, as well as an explanation of any material
changes from previous plans. The information should also contain an
analysis of the sensitivity of the cost figures to specific
assumptions, including a range of values for key cost assumptions,
their impact on total cost estimates of the census, the likelihood
associated with those ranges, and their impact on the total estimated
cost of the census.
As part of this process, OMB should establish triggers that would
signal when the yearly 2010 Census costs, total 2010 Census costs, or
both exceeded some predetermined amount. In such instances, the Bureau
should then be required to prepare a special report to Congress and OMB
justifying why the additional costs were necessary and what
alternatives were considered.
Further, to enhance the Bureau's performance and accountability, as
well as to help convince Congress and other stakeholders that the
Bureau has chosen an optimum design and will manage operations and
control costs effectively, we recommend that the Secretary of Commerce
direct the Bureau to improve the rigor of its planning process by
developing an operational plan that consolidates budget,
methodological, and other relevant information about the 2010 Census
into a single, comprehensive project plan that would be updated as
needed. Individual elements could include:
* specific performance goals, how the Bureau's efforts, procedures, and
projects would contribute to those goals, and what performance measures
would be used;
* risk and mitigation plans that fully address all significant
potential risks;
* detailed milestone estimates that identify all significant
interrelationships; and:
* annually updated life cycle cost estimates, including a sensitivity
analysis, and an explanation of significant changes in the assumptions
on which these costs are based.
Moreover, to help ensure that the nation has at its disposal the best
possible data should there be problems with the quality of 2010 Census,
the Bureau, with input from both majority and minority parties in the
Senate and House of Representatives, should research the feasibility of
procedures that could allow it to adjust census results for those
purposes for which it is both legal and appropriate to do so and, if
found to be feasible, test those procedures during the 2006 census test
and 2008 census dress rehearsal.
Agency Comments and Our Evaluation:
The Secretary of Commerce forwarded written comments from the U.S.
Census Bureau on a draft of this report that we received November 6,
2003. The comments are reprinted in appendix I. The Bureau generally
disagreed with many of our key findings, conclusions, and
recommendations.
The Bureau believes that the report, in its discussion of escalating
census costs, ignores the fact that a key cost driver is stakeholders'
demand for better accuracy. We agree with the Bureau that its mandate
to count each and every resident in the face of countervailing societal
trends is an important reason for the cost increases. As we noted in
the report, societal changes have reduced the cost-effectiveness of the
census, and it has become more and more difficult to stay on par with
the results of previous enumerations. Similarly, we stated that "the
legal requirement to count everyone leads the Bureau to employ
operations that only marginally improve coverage but that are
relatively expensive to conduct.":
Further, we do not, as the Bureau asserts, treat the cost issue in a
vacuum, and agree with the Bureau that little would be gained by
focusing on the cost of the 2010 Census alone. Rather, any
deliberations on the 2010 Census need to focus on how changes in
spending on the census might affect the quality of the count. Our draft
report emphasized this exact point noting that "The growing cost of the
head count, at a time when the nation is facing historic budget
deficits, highlights the importance of congressional deliberations on
the extent to which each additional dollar spent on the census results
in better data, as well as how best to balance the need for a complete
count, with the need to ensure the cost of a complete count does not
become unreasonable." Similarly, we concluded that "it will also be
important for policymakers to consider, early in the decade, the long-
term costs associated with the census and finding the right balance
between controlling mushrooming costs and improving accuracy.":
The Bureau also believes the report implies that the cost increases are
caused by the reengineering effort. Our draft report did not state, nor
did we intend to imply, that the reengineering effort would cause most
of the projected cost increases for the 2010 Census. In fact, our
report even notes that the Bureau's reengineering strategy has the
potential to reduce costs relative to a design that would repeat the
Census 2000 approach. To help clarify this point, we added text that
describes how a repeat of the 2000 approach would be more costly than
the reengineered design, according to Bureau estimates.
The Bureau disagreed with our recommendation to OMB regarding the need
for greater budgetary transparency, noting that the real reason for the
vagueness of out-year cost estimates stems from a fundamental
difference of opinion between the administration and Congress over the
appropriate time to share that information. We believe that it is
important for the administration to provide details of out-year cost
projections for the decennial census for the reason stated in our draft
report: annual budget requests and reports provided to Congress early
in the decennial census life cycle do not reflect the full cash
consequences of the spending in later years of the decade. We
understand that the Bureau has followed the administration's guidance
on providing out-year cost estimates; this is also why we directed our
recommendation for greater fiscal transparency to OMB, which we discuss
in greater detail below.
The Bureau disagreed with our recommendation to improve the rigor of
its planning process by developing an operational plan that
consolidates budget, methodological, and other relevant information
into a single, comprehensive project plan. The Bureau noted that these
documents already exist and are widely available, and the Bureau
already shares them with Congress, us, the National Academy of Sciences
(NAS) Panel on Research on Future Census Methods--the panel responsible
for reviewing the census, and other stakeholders.
While we agree with the Bureau that some of this information is
available (and we noted this fact in our draft report), it is
piecemeal--one can only obtain it by cobbling together the Bureau's
budget submission, its strategic plan, and several other documents, and
even then, key information such as performance goals would still be
lacking. Further, although the Bureau notes that it has provided this
information to the NAS panel, as we stated in our report, NAS, like us,
also found the information wanting. As we described in the report, the
panel shared our concerns over the need for a coherent project plan,
and called on the Bureau to develop a business plan that among other
things, itemized the goals, costs, and benefits of each census
component and a describes how they contributed to a high-quality
census. Whether it is called a business plan or a project plan, such
information is not, as the Bureau maintains, simply "more process."
Quite the contrary, this information is essential for improving
performance; facilitating a thorough, independent review of the
Bureau's plans; and demonstrating to Congress and other stakeholders
that the Bureau can effectively design and manage operations and
control costs.
The Bureau incorrectly asserts that our report criticizes it for not
completing the evaluations of the 2000 Census in a timely manner. Our
report did not address this matter, although NAS's Second Interim
Report on Planning the 2010 Census urged the Bureau to "give high
priority to evaluation studies" and complete them as expeditiously as
possible. We agree that the Bureau's planning staff do have access to
the draft evaluations, and in fact, we noted in the report that they
are using them in planning for the 2010 Census.
The key point, however, is the Bureau's ability to build on the results
of the 2000 Census. This could be hampered by the fact that while the
evaluation program assessed numerous aspects of the census, the Bureau
still lacks data on the performance of key census activities as well as
how specific census operations contributed to two of the Bureau's key
goals for 2000: improved accuracy and cost-effectiveness.
The Bureau agreed with us that it is important to bring closure to the
discussion on whether and how coverage measurement will be used in the
2010 Census. However, the Bureau believes that the approach used for
the 2000 Census proved that it was not feasible to produce a final
analysis of coverage measurement in time to meet redistricting
requirements. We agree with the Bureau's assessment that the coverage
measurement approach used in the 2000 Census needs to be reworked.
However, this should not preclude it from researching alternative
approaches for the 2010 Census in light of the fact that the Bureau's
ability to maintain the level of quality reported for the 2000 Census
is less than certain.
Finally, the Bureau questioned our assessment that the only contingency
plan for conducting the 2010 Census, if the reengineered effort fails,
was to fall back on the Census 2000 methods. The Bureau maintains that
the 2000 Census was by most accounts a very successful census and,
accordingly, the Bureau already has available the methods and
procedures for taking an excellent census. Our report does not advocate
the development of another set of census methods. Rather, we were
trying to illustrate the challenge the Bureau faces in implementing its
reengineering plans, where the failure of any one leg could compromise
the other two, thereby requiring the Bureau to rely on the approach it
used for the 2000 Census. According to Bureau officials, this in turn
could make it difficult for the Bureau to accomplish its goals for the
2010 Census, which include cost containment and better quality data.
On October 14, 2003, the Associate Director for General Government
Programs, OMB, provided written comments on a draft of this report,
which are reprinted in appendix II. OMB shared our view that the costs
and risks associated with the 2010 Census must be carefully monitored
and evaluated throughout the decade. OMB also agreed that it is
essential to understand the key cost drivers and said that it is
working with the U.S. Census Bureau to ensure that the Bureau develops
high-quality, transparent life cycle cost estimates.
However, OMB disagreed with our recommendation that it ensure that the
Bureau, include a notational item in the Program and Financing (P&F)
schedule of the President's Budget with an updated estimate of the life
cycle costs of the census and the amount of money the Bureau expects to
spend in each year of the cycle, as well as an explanation of any
significant changes from previous plans. OMB believes that the Bureau's
report on the life cycle costs, which is updated regularly, is the best
mechanism to present estimates of the total life cycle costs and
explanations for any material changes from previous plans. Further, OMB
noted that presenting this information in the P&F schedule is
cumbersome and unnecessary because the Analytical Perspectives volume
of the President's Budget currently shows out-year estimates that
incorporate anticipated programmatic changes of the Decennial Census
within the Periodic Censuses and Program account.
As noted in our report, we do not believe the information OMB currently
reports to Congress is sufficiently timely or detailed to provide the
level of transparency needed for effective congressional oversight and
cost control. Indeed, while OMB cites the Bureau's life cycle report,
the document that we reviewed for this report took the Bureau nearly 2
years to revise. Moreover, the revised estimates, like the original
estimates, overstated the life cycle cost estimate by $300 million
because the Bureau did not take into account a surplus of that amount
that it identified near the end of fiscal year 2000. Although the
Bureau is to reissue the 2010 life cycle cost estimates early in
calendar year 2004, the incorrect estimates will have been in
circulation for more than 2 years.
Additionally, the information contained in the Analytical Perspectives
volume of the President's Budget is limited. For example, it only
provides information on out-year estimates for 5 years. As a result,
the volume will not include estimates for the high-cost year of 2010
until the release of the President's fiscal year 2006 budget. Further,
the Analytical Perspectives volume lacks information on the sensitivity
of cost figures to specific assumptions and the likelihood of these
estimates. It also does not contain any explanations of changes in cost
estimates from year to year. Complete and transparent information on
out-year costs is important to inform deliberations on the
acceptability of these costs and to ensure that Congress understands
the possible range of census life cycle costs.
OMB also disagreed with our recommendation to establish triggers to
signal when the yearly 2010 Census costs, total 2010 Census costs, or
both exceeded some predetermined amount. OMB noted that it has
established internal procedures within its budget reviews to monitor
2010 Census costs and believes they are sufficient for ensuring that
estimates are not exceeded without clear justification. OMB added that
this justification could be included in the Bureau's updates to its
life cycle cost estimates.
Although OMB's internal procedures might be sufficient for OMB's
requirements, they do little to address the fundamental need for
greater fiscal transparency. Continued reliance on these procedures
would inhibit independent review by congressional and other external
stakeholders, as well as limit informed discussion of the trade-offs of
dollars versus accuracy and what cost control measures, if any, might
be needed to make the 2010 Census more affordable.
In closing, OMB commented that the Bureau's reengineering plan is being
reviewed by NAS as well as seven advisory committees. OMB stated that
the analyses stemming from these reviews, such as NAS's recently issued
report on census planning, enhance the Bureau's accountability and help
ensure "that the ultimate 2010 Census design is optimal.":
We agree with OMB that NAS and the advisory committees are important
for reviewing the Bureau's plans and holding the Bureau accountable for
a cost-effective census in 2010. And this is precisely why we made the
recommendations that we did. Without a transparent budgeting and
planning process, a thorough, independent review by these and other
external groups would be difficult to impossible. That greater
transparency is needed in both these areas is highlighted not just in
our report, but in the very NAS study that OMB cites. Indeed, NAS found
that "a major conclusion of the panel is that discussion of the 2010
Census design needs to be more fully informed by the evaluation of
various trade-offs--the costs and benefits of various reasonable
approaches in order to make wise decisions."[Footnote 24]
As agreed with your offices, unless you release its contents earlier,
we plan no further distribution of this report until 30 days from its
issue date. At that time we will send copies to other interested
congressional committees, the Secretary of Commerce, the Director of
the U.S. Census Bureau, and the Director of the Office of Management
and Budget. Copies will be made available to others upon request. This
report will also be available at no charge on GAO's Web site at
[Hyperlink, http://www.gao.gov] http://www.gao.gov.
If you or your staff have any questions concerning this report, please
contact me at (202) 512-6806 or [Hyperlink, daltonp@gao.gov]
daltonp@gao.gov or Robert Goldenkoff, Assistant Director, at (202) 512-
2757 or [Hyperlink, goldenkoffr@gao.gov] goldenkoffr@gao.gov. Key
contributors to this report were Richard Donaldson, Ty Mitchell, Robert
Yetvin, and Christine Bonham.
Signed by:
Patricia A. Dalton:
Director:
Strategic Issues:
[End of section]
Appendixes:
Appendix I: Comments from the Department of Commerce:
THE SECRETARY OF COMMERCE
Washington. D.C. 20230:
October 9, 2003:
Ms. Patricia A. Dalton
Director, Strategic Issues
U.S. General Accounting Office
Washington, DC 20548:
Dear Ms. Dalton:
The U.S. Department of Commerce appreciates the opportunity to comment
on the U.S. General Accounting Office draft report entitled 2010
Census: Cost and Design Issues Need to Be Addressed Soon. I enclose the
Department's comments on this report.
Sincerely,
Signed by:
Donald L. Evans:
Enclosure:
Enclosure:
Comments from the U.S. Department of Commerce, U.S. Census Bureau,
Regarding the U.S. General Accounting Office (GAO) Draft Report
Entitled 2010 Census: Cost and Design Issues Need to Be Addressed Soon:
General Comments on the Report:
Thank you for providing the opportunity to comment on your draft
report, 2010 Census: Cost and Design Issues Need to Be Addressed
Soon.":
First, the discussion about the escalating costs of census programs
over recent decades completely ignores one of the major drivers of cost
increases since the inception of revenue sharing in 1970-namely, the
ever-increasing demands for improved coverage accuracy made by the
Congress; state, tribal, and local governments; advisory committees;
and other stakeholders. If the GAO believes the Congress and others
need to have a serious conversation about whether this Nation can
afford to spend over $11 billion for the next decennial census
program, then the conversation must begin by asking if "perfect"
coverage still must be the goal. Very little will be gained from these
conversations if cost is treated in a vacuum or as the only dimension.
Second, the Bureau is concerned with the fact that the report's
discussion of the estimated $11.3 billion cost for the reengineered
2010 census program is presented in a way that implies that much of the
increase from 2000 will be caused by the reengineering effort. In fact,
the Bureau currently estimates that repeating the Census 2000 approach
would cost much more-$12.2 billion--and that the reengineering effort
will reduce this by nearly $1 billion.
Third, the Bureau disagrees with most of the report's assertions about
an alleged lack of transparency in budget details for later years in
the decade. We believe these assertions are misdirected-the underlying
issue stems from a fundamental difference of opinion between the
Administration and the Congress about the appropriate time to share
detailed outyear cost estimates. In this same vein, the Bureau
disagrees with the report's recommendations regarding the Director of
the Office of Management and Budget (OMB) suggesting that we display
updated estimates of census life-cycle costs and the amount of money
that we expect to spend in each year of the cycle, as a note in the
Program and Financing schedule of the President's budget. This
recommendation implies that the Bureau is unwilling to share such
outyear cost information.
Fourth, the Bureau strongly disagrees with the report's finding that
the 2010 planning process is somehow lacking because all of the key
documents about goals, assumptions, and plans are not combined in a
single, comprehensive document. These key documents exist and are
widely available; the Census Bureau has shared them with the GAO, the
Congress, the OMB, advisory committees, and any other stakeholders
expressing interest in these matters. Recipients include the National
Academy Panel mentioned in the report, which received these documents
earlier this year.
The Bureau is confident that no prospective census in history has ever
been as thoroughly documented regarding goals and methods this early in
the decade. We thus disagree with GAO's recommendation for the
Secretary of Commerce to establish a new documentation process to
ensure that the Department and the OMB are kept informed of plans and
progress about the 2010, census program. Such a process already exists,
and the Secretary and the OMB already receive a great deal of
information each year through that process, primarily via the annual
budget submission (including Exhibit 300,
Capital Asset Plan and Business Case for information technology),
through regular updates to the Department's and Census Bureau's
Strategic Plan, through the Annual Performance Plan process, and now
through the OMB's new Program Assessment and Rating Tool process. Yet
more "processes" would only serve to delay and to increase the costs of
2010 census planning.
Fifth, the report criticizes the Census Bureau because a number of
evaluations from Census 2000 have not been completed. The report
asserts that these evaluations cannot be used to inform decisions for
the 2010 census. However, the fact that a report is not final or
published does not mean the information is unavailable to Census Bureau
staff involved with the 2010 census planning efforts. Planning staff do
have access to draft reports. In addition, many of these staff members
were directly involved with the same operations in 2000, so they know
many of the issues and experiences firsthand. It should also be noted
that the Bureau conducted many more evaluations than in previous
decades, yet has completed the evaluation program earlier than in any
of those same decades.
Sixth, the Census Bureau plans to conduct research on coverage
measurement methodologies for use in 2010. It also plans to test those
methods in the 2006 Census Test and then use them in the 2008 Dress
Rehearsal. As noted in the report, however, the Census Bureau believes
it is not feasible to produce final analyses of coverage measurement
data in time to meet legal deadlines. Further, the statutory deadline
itself is not the only impediment to adjusting the census. Experience
with Census 2000 demonstrated both that full assessment of the Accuracy
and Coverage Evaluation _ (A.C.E.) data required time far beyond that
required by states to meet redistricting requirements under any
realistic scenario, and that the difficulties in uses of coverage
measurement methodologies to improve the accuracy of census counts at
the tract level may well be insurmountable.
We agree with the GAO that it is important to bring to closure at an
early time the discussion on whether and how coverage measurement will
be used in the 2010 census. An early conclusion of this matter is
important. Therefore, the Director has spoken with interested Members
of Congress, and the Bureau has clearly indicated through its budget
process that it does not plan to seek funds for developing a coverage
measurement procedure designed to be used to adjust data for
redistricting purposes. The Census 2000 and A.C.E. was a definitive
test of this approach and provided more than ample evidence that this
goal cannot be achieved.
Finally, the Bureau is troubled by the report's assertion that the only
contingency plan for conducting the reengineered 2010 census (that
incorporates the American Community Survey, a GPS-aligned and
modernized MAF/TIGER, and a short-form-only questionnaire) seems to be
"falling back" to Census 2000 methods. Census 2000 was by most accounts
a very successful census. Accordingly, the Bureau already has available
the methods and procedures for taking an excellent census. As part of
the reengineering effort, the Bureau is incorporating lessons learned
from the Census 2000 evaluations and investigating improvements to
those methods. There is no need, and certainly no time or resources
available, to develop yet another set of census methods--different from
those used in Census 2000-to serve as a contingency plan for the
reengineering approach.
[End of section]
Appendix II: Comments from the Office of Management and Budget:
EXECUTIVE OFFICE OF THE PRESIDENT
OFFICE OF MANAGEMENT AND BUDGET
WASHINGTON, D.C. 20503:
October 14, 2003:
Ms. Patricia Dalton
Director:
Strategic Issues Team:
US General Accounting Office
Washington, DC 20548:
Dear Ms. Dalton:
Thank you for the opportunity to comment on the draft report titled,
"2010 Census: Cost and Design Issues Need to be Addressed" (GAO-04-37).
The 2010 Census will be an important investment, and we share GAO's
view that the costs and risks must be carefully monitored and evaluated
throughout the decade. In addition, we agree that it is essential to
understand the key cost drivers and we are working with the Census
Bureau to ensure development of high-quality, transparent, life cycle
cost estimates.
We believe the Census Bureau's report on the life cycle costs, which is
updated regularly, is the best mechanism to present updated estimates
of the total life-cycle cost and an explanation of any material changes
from previous plans. Presenting this information in the Budget's
Program and Financing (P&F) schedules is cumbersome and unnecessary.
The Analytical Perspectives volume of the President's Budget currently
shows out-year estimates that incorporate anticipated programmatic
changes of the Decennial Census within the Periodic Censuses and
Programs (PC&P) account. These out-year estimates are not considered
binding by the Administration. However, they serve as a general
guideline for future Budget requests when the Administration could be
faced with different resource constraints and national priorities.
We also do not agree that establishing triggers within the President's
Budget is appropriate for this program. We have established internal
procedures within our budget reviews to carefully monitor 2010 Census
costs and believe they are sufficient for ensuring that estimates are
not exceeded without clear justification. Further, this justification
could be included within the Census Bureau's updates to the life cycle
cost:
report. In addition, the Census Bureau's reengineering plan is being
reviewed by the National Academy of Sciences as well as seven advisory
committees --the Decennial Census Advisory Committee, five separate
Race and Ethnic Advisory Committees, and the Census Advisory Committee
of Professional Association. Analyses stemming from these reviews, such
as the National Research Council's recently issued Planning the 2010
Census, Second Interim Report, serve to enhance the Bureau's
accountability and to help ensure that the ultimate 2010 Census design
is optimal.
Thank you again for providing us with a draft copy of the report to
review and comment upon.
Sincerely,
Signed by:
Stephen S. McMillin:
Associate Director for General Government Programs:
[End of section]
Appendix III: 2010 Census Planning Organization:
The U.S. Census Bureau (Bureau) has given its Decennial Management
Division responsibility for planning the 2010 Census, including the
American Community Survey (ACS) and the Master Address File/
Topologically Integrated Geographic Encoding and Referencing (MAF/
TIGER) enhancements. The division established an executive steering
committee, shown in figure 5, for this purpose.
Figure 5: Decennial Management Division's Organizational Structure for
Planning the 2010 Census:
[See PDF for image]
[End of figure]
Each group under the 2010 Census planning organization has specific
activities that it was charged with studying. Listed below are the
research and development planning groups chartered at the time of our
review and a list of these activities.
2010 Census Reengineering: Research and Development Planning Groups:
The Content Planning Group:
* determines alternatives for question wording and order;
* develops consistent questions across multiple response modes;
* develops consistent forms design across multiple response modes;
* researches use of administrative records for missing data;
* researches and documents analysis of tabulated items' quality,
including item response rate for misreported, underreported, or
generally nonreported questions;
* researches other methods of imputing characteristics;
* researches information on administrative records and aggregate census
tract modeling;
* explores attitudinal trends on the mailing package and questionnaire;
* develops alternative formats for race and Hispanic origin question
layout;
* researches printing alternatives and new technologies;
* researches United States Postal Service requirements and alternative
services related to addressing, barcodes, and delivery;
* coordinates content development and congressional approval of ACS;
* assesses appropriate questionnaire wording to ensure conformance with
the enumeration and residence rules; and:
* defines and assesses appropriate question wording for "usual home
elsewhere" on any self-response and enumerator instrument(s).
The Special Place/Group Quarters Planning Group:
* reviews definitions of housing units and group quarters, including
transient locations, and where necessary, modifies the definitions to
reflect current living arrangements to reduce census classification
errors;
* determines the best methods for implementing housing unit and group
quarters definitions in 2010 Census operations that reduce decennial
census classification errors;
* determines what operations are required to improve the accuracy of
the current Census 2000 inventory file and the geographic reference
base, including geographic assignment;
* determines what coverage improvement operations are required to
ensure complete coverage of special places and group quarters;
* develops a process that ensures the integration of special place/
group quarters addresses with housing unit addresses and subsequent
updates into the MAF/TIGER system;
* determines the appropriate enumeration methods for the group quarters
population;
* determines the appropriate enumeration method for embedded housing
units within special place/group quarters that will reduce 2010 Census
error;
* determines improvements to the enumeration process, including
appropriate requirements regarding the implementation of mobile
computing devices;
* determines questionnaire requirements;
* assesses the quality of administrative records for the group quarters
population and determines how and if administrative records should be
incorporated into the enumeration process; and:
* determines privacy issues, legal issues, or both and solutions and
identifies a process to implement these findings into operational
procedures.
The Self Response Options Planning Group:
* identifies the optimal mix of options, including mail, Internet, and
telephone, by which the public can respond to the census;
* identifies options for accommodating responses by people who believe
they have not been counted by the traditional methods;
* identifies messages and methods to inform, advertise, and motivate
census questionnaire response in a multimode self-response environment;
* determines the optimal mailing strategy;
* identifies methods and options for delivering a replacement
questionnaire;
* examines methods for making the identification number shorter and
easier to handle;
* develops an integrated self-response that optimizes requirements from
other program areas, such as language and self-initiated response
requirements; and:
* develops strategies for accommodating response by people with
disabilities.
The Overseas Enumeration Planning Group:
* makes recommendations to the Bureau's executive staff on key policy
issues, such as applying residence rules overseas, goals of counting
Americans living overseas, the definition of this universe, and
conducting a count that would take a different form than the stateside
enumeration;
* determines possible methods for enumerating Americans overseas in
accordance with decisions on the policy issues;
* determines the type of evaluations that will be conducted to assess
the quality of data captured during this operation; and:
* defines a process and plan for consultations with stakeholders.
The Field Activities Planning Group:
* determines the optimal fit of new technologies, such as mobile
computing devices, within the enumeration methods and processes;
* determines the optimal and most cost-effective field infrastructure
to support field activities;
* determines the possible enumeration methods, such as Update/Leave,
Urban Update/Leave, and Update/Enumerate and their feasibility for the
2010 short-form-only census;
* determines how to apply the criteria established for the areas in
which each type of enumeration method will be used;
* determines how best to target for special methods areas where mail
delivery methods are not optimal, for example, areas where residents
use post office boxes;
* determines the most effective special methods or tool kit approaches
for enumerating housing units in hard to enumerate areas, such as gated
communities;
* determines the most efficient use of the telephone data collection
methodology to reduce personal field visits;
* determines the most effective management and implementation approach
that will ensure integrated quality assurance in field activities;
* determines the best methodology for delineating assignment areas;
and:
* determines data requirements for field instruments relating to
management and quality control needs.
The Coverage Planning Group:
* proposes content rules for identifying which addresses to include in
the 2010 Census;
* proposes consistent and accurate approaches for improving matching
rules to reduce duplication of addresses;
* determines the appropriate use of administrative records for MAF
activities, such as targeting and/or updating MAF;
* determines areas in which MAF and TIGER need improvement;
* determines best methods for targeting areas that need updating;
* develops requirements for the positional accuracy of MAF/TIGER in
coordination with the Global Positioning System Applications Strategy
Group;
* defines and assesses methods to improve the address list and coverage
of the 2010 Census after mail-out address file is final;
* defines criteria for determining the areas in which each type of
enumeration method will be used;
* in coordination with the Special Place/Group Quarters Planning Group,
develops a process that ensures the integration of special place/group
quarters addresses with housing unit addresses and subsequent updates
of the MAF/TIGER system;
* reviews the enumeration and residence rules, taking into
consideration implementation issues, and recommends appropriate
revisions;
* defines and assesses appropriate methods for dealing with usual home
elsewhere situations on all questionnaires and instruments;
* determines methods for avoiding multiple listings of persons within
and between households;
* determines methods for improving within household coverage, including
the Be Counted program;
* develops thresholds reasonable for count imputations; and:
* determines best count imputation method, including the use of spatial
analysis tools, to determine locational relationships.
The Software and Systems Planning Group:
* determines acceptable industry standards and practices for the
software process;
* identifies and assesses alternative standards and practices for
decennial applications for requirements development, software analysis
and design, testing methods, quality assurance, configuration
management, documentation of all components, and risk management;
* recommends standards and practices for the software process;
* identifies and assesses the need for a common data dictionary,
including integrating it with the corporate meta-data project;
* recommends whether to develop a common data dictionary solution, and
if adopted, develops a rationale for its use and defines its functional
requirements;
* reviews deliverables and recommends requirements for the Technical
Architecture and Infrastructure contract in preparation for development
of the 2010 Census systems architecture;
* evaluates potential systems components;
* recommends approach for developing and selecting the 2010 Census
physical architecture;
* recommends physical architecture solution;
* for mobile computing devices (MCD),
* determines which operations will test the use of the MCDs,
* determines what functionality will be employed in MCDs for testing,
* determines which specific type(s) of MCDs and which software
applications the Bureau will test, and:
* considers the implications for field infrastructure, particularly
with respect to the technology needed to support MCD applications;
* for Internet response mode,
* tests alternatives for validating/confirming receipt of census
questionnaire/response,
* researches and tests mechanisms for respondents to generate e-mail to
the Bureau,
* identifies options for e-mail to respondents with "census message,"
questionnaire, or both,
* researches and tests alternatives for serving multiple forms (such as
those needed for the language program and overseas enumeration),
* researches and tests use of census housing unit identification,
* determines effective ways to maintain security,
* researches and tests alternative interfaces, and:
* identifies alternatives for meeting accessibility needs;
* for high technology data collection modes,
* researches ways to utilize Web television,
* researches ways to utilize kiosks, and:
* identifies alternatives for meeting accessibility needs; and:
* for telephone response modes,
* researches and tests alternatives for initial response by voice
recognition/IVR/touch tone/agent assisted,
* tests alternatives for automated dial-up "census message,":
* researches and tests alternatives for languages,
* determines effective ways to maintain security, and:
* identifies alternatives for meeting accessibility needs.
The Communications Planning Group:
* determines how the public will be divided into groups in order to
target specific messages;
* determines the most appropriate messages that will resonate with the
Bureau's audiences and motivate them to respond and the strategies for
creating a brand (name/image) for the Bureau;
* develops innovative strategies using 2000 Census and ACS data to
reach the:
* hard to count population,
* emerging populations,
* rural communities, and:
* non-English-speaking populations;
* develops media strategies and messages for media outlets, operations
that need specific advertising or publicity, and crisis communications
management;
* considers 2010 Census cost implications; and:
* determines the best methods for evaluating the effectiveness of the
components of the integrated marketing strategy.
The Language Programs Planning Group:
* researches the availability, design, and use of bilingual forms;
* researches and develops alternative operational testing requirements
for:
* alternatives for an English/Spanish mail-out questionnaire,
* the languages the 2010 Census will support,
* questionnaire assembly,
* process for requesting a questionnaire in a specific language,
* questionnaire/guide distribution,
* validation of a bilingual questionnaire,
* alternatives for a bilingual enumerator instrument,
* questionnaire data capture,
* data entry for all collection modes,
* accessibility for in-language respondents (Braille, audio, etc.),
and:
* partnership responsibilities; and:
* considers Advisory Committee lessons learned and recommendations.
The Race and Ethnicity Planning Group:
* determines if revised instructions to the questions on race and
Hispanic origin convey the federal government's distinction of race and
Hispanic origin as two separate concepts that require responses that
reflect the intent of each question;
* determines alternative wording to the questions on race and
ethnicity;
* determines the type of analyses and evaluations needed to assess the
quality of race and ethnicity data from the 2000 Census and ACS;
* determines if the exclusion of a "Some other race" response category
in the question on race will affect race reporting;
* determines if changing the order of the race response categories
affects race reporting;
* determines what effect the inclusion of examples for "Other Asian"
and "Other Pacific Islander" have on race reporting;
* determines if the use of shortened response categories during
telephone and other data collection modes will affect respondents'
understanding of the response options;
* determines whether alternative terminology (e.g., "Print race or
races" instead of "Print category or categories") will affect race
reporting;
* determines whether a revised Hispanic question that does not include
"Spanish" as an identifier will increase reporting of specific Hispanic
origin responses in the "Other Hispanic" response category; and:
* determines whether the inclusion of examples for detailed Hispanic
groups will increase specified responses in the "Other Hispanic"
response category.
The American Indian and Alaska Native (AIAN) Planning Group:
* determines what enumeration methods will be tested in AIAN areas;
* determines outreach and partnership strategies for testing in AIAN
areas;
* determines strategies and explores methodologies for improving
coverage; and:
* assesses developments or innovations needed for mapping and
enumeration activities.
(450178):
FOOTNOTES
[1] See, for example, U.S. General Accounting Office, 2000 Census:
Progress Made on Design, but Risks Remain, GAO/GGD-97-142 (Washington,
D.C.: July 14, 1997).
[2] TIGER is a registered trademark of the U.S. Census Bureau.
[3] U.S. General Accounting Office, 2000 Census: Lessons Learned for
Planning a More Cost-Effective 2010 Census, GAO-03-40 (Washington,
D.C.: Oct. 31, 2002).
[4] See, for example, U.S. General Accounting Office, 2000 Census:
Progress Made on Design, but Risks Remain, GAO/GGD-97-142 (Washington,
D.C.: July 14, 1997).
[5] National Research Council, Planning the 2010 Census: Second Interim
Report (Washington, D.C.: July 22, 2003).
[6] The guides include: A Guide to the Project Management Body of
Knowledge, Project Management Institute Standards Committee, „ 1996.
Project Management for Mission Critical Systems, a Handbook for
Government Executives, Information Technology Resource Board, April 5,
2001.
Capability, Maturity Model Integration Project Planning Guide, Carnegie
Mellon Software Engineering Institute, March 2001.
U.S. General Accounting Office, Business Process Reengineering
Assessment Guide, GAO/AIMD-10.1.15 (Washington, D.C.: May 1997).
[7] U.S. General Accounting Office, Major Management Challenges and
Program Risks: Department of Commerce, GAO-03-97 (Washington, D.C.:
January 2003).
[8] Department of Commerce vs. United States House of Representatives,
525 U.S. 316 (1999).
[9] For a more complete discussion of ACS, see U.S. General Accounting
Office, The American Community Survey: Accuracy and Timeliness Issues,
GAO-02-956R (Washington, D.C.: Sept. 30, 2002).
[10] Department of Commerce v. United States House of Representatives,
525 U.S. 316 (1999).
[11] U.S. General Accounting Office, 2000 Census: Coverage Measurement
Programs' Results, Costs, and Lessons Learned, GAO-03-287 (Washington,
D.C.: Jan. 29, 2003).
[12] These figures include the 10-year costs for ACS replacement for
the census long form and the costs of MAF/TIGER.
[13] U.S. General Accounting Office, Fiscal Exposures: Improving the
Budgetary Focus on Long-Term Costs and Uncertainties, GAO-03-213
(Washington, D.C.: Jan. 24, 2003).
[14] OMB Circular A-94 provides guidelines for cost-benefit analysis of
federal programs and recommends that agencies develop a sensitivity
analysis for major projects with significant uncertainty. The circular
provides a method for determining how sensitive outcomes are to changes
in assumptions.
[15] A lower mail-back response rate means that more costly nonresponse
follow-up activities are needed to obtain census data from a household.
[16] U.S. General Accounting Office, Managing for Results: Efforts to
Strengthen the Link Between Resources and Results at the Nuclear
Regulatory Commission, GAO-03-258 (Washington, D.C.: Dec. 10, 2002).
[17] Details are documented in U.S. Census Bureau, Decennial Management
Division, Reengineering the Decennial Census: The Vision of a Baseline
Design for 2010 (Washington, D.C.: January 2003).
[18] U.S. Department of Commerce, Office of Inspector General,
Improving Our Measure of America: What Census 2000 Can Teach Us in
Planning for 2010, OIG-14431 (Washington, D.C.: Spring 2002).
[19] OMB Circular A-11, as revised July 2003, provides guidance to
agencies on budget preparation, performance reporting, and capital
asset acquisition.
[20] For more information on the difficulties the Bureau encountered
generating reliable productivity data, see U.S. General Accounting
Office, 2000 Census: Better Productivity Data Needed for Future
Planning and Budgeting, GAO-02-4 (Washington, D.C.: Oct. 4, 2001).
[21] For example, if an address is noted to be vacant, an error message
will appear if the enumerator attempts to count people at that address.
[22] Exhibit 300 is a capital asset plan required under OMB Circular A-
11, Section 300.
[23] U.S. Census Bureau, Decennial Management Division, 2010 Decennial
Census Risk Management Plan (Washington, D.C.: Mar. 4, 2003).
[24] National Research Council, Panel on Research on Future Census
Methods, Committee on National Statistics, Planning the 2010 Census:
Second Interim Report (Washington, D.C.: 2003).
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
