2010 Census
Bureau Needs to Specify How It Will Assess Coverage Follow-up Techniques and When It Will Produce Coverage Measurement Results Gao ID: GAO-08-414 April 15, 2008An accurate decennial census relies on finding and counting people--only once--in the right place and collecting complete and correct information. GAO reviewed the Department of Commerce's U.S. Census Bureau's (Bureau) plans to (1) improve the census's accuracy--or coverage--with its coverage follow-up operation, (2) assess the accuracy of the census through its census coverage measurement program, and (3) evaluate these efforts after 2010. GAO reviewed documentation and interviewed Bureau officials about the Bureau's coverage follow-up and measurement plans. GAO compared plans for measuring coverage in 2000 and 2010 and interviewed 15 experts about changes to the plans.
The Bureau plans to expand its coverage follow-up operation to improve the accuracy of enumeration data collected in 2010. As in Census 2000, coverage follow-up is intended to resolve count discrepancies (when the number of household members reported on a completed questionnaire fails to match the number of persons for whom information is collected) and complete the enumeration of large households (more than six persons living in a housing unit). For 2010, the Bureau researched new techniques to increase the number of ways a person or household could be selected for coverage follow-up. These include (1) using coverage probes on the census form to identify cases with potential under- or overcount problems, (2) using administrative records to identify households that include persons who may not have been counted, and (3) using computer matching to determine potential duplicate persons. However, it is not clear on what basis the Bureau will select or prioritize cases from this potentially expanded universe. The Bureau plans to further assess the techniques during the 2008 Dress Rehearsal, leaving little time to consider and implement improvements for the 2010 coverage follow-up operation. Because of the potential for the coverage follow-up workload to increase, it will be important for the Bureau to be clear on how it will decide which techniques to use and how it will select coverage follow-up cases to ensure that the operation may best improve coverage. Although the Bureau's overall design for measurement is similar in many respects to the design for 2000, its measurement plan differs in important ways. The Bureau is planning to use a sample design similar to that used in 2000 and still intends to produce net coverage error estimates. However, the Bureau does not plan to use measurement results to adjust the 2010 count of the nation's population and is planning to make four significant changes: (1) conduct the measurement interview 4 months later than it did in 2000, (2) use a different method--logistic regression--to estimate net error, (3) use estimates of the components of coverage error in an effort to improve future decennials, and (4) expand computer matching and clerical matching to match persons in its postenumeration survey to the entire census. Still, the Bureau has not specified when it will produce estimates of net coverage error and the components of coverage error for the 2010 Census. The Bureau also has not clearly described how it will associate the components of coverage error to census operations in order to improve future decennials. Because of the Bureau's changes to its coverage measurement program, it is important for the Bureau to solidify its plans and share them with stakeholders to ensure that there is an agreement on what the program will produce and when the Bureau will produce its results. The Bureau is in the early stages of planning how it will evaluate coverage follow-up and measurement after the 2010 Census. As part of this effort, the Bureau is considering recommendations of the National Academy of Sciences for the Bureau's program of evaluations and experiments for Census 2010. The Bureau has not yet developed a timeline for completing its evaluation plans.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-08-414, 2010 Census: Bureau Needs to Specify How It Will Assess Coverage Follow-up Techniques and When It Will Produce Coverage Measurement Results
This is the accessible text file for GAO report number GAO-08-414
entitled '2010 Census: Bureau Needs to Specify How It Will Assess
Coverage Follow-up Techniques and When It Will Produce Coverage
Measurement Results' which was released on May 17, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Committee on Oversight and Government Reform,
House of Representatives:
United States Government Accountability Office:
GAO:
April 2008:
2010 Census:
Bureau Needs to Specify How It Will Assess Coverage Follow-up
Techniques and When It Will Produce Coverage Measurement Results:
Improving and Measuring Census Coverage:
GAO-08-414:
GAO Highlights:
Highlights of GAO-08-414, a report to the Chairman, Committee on
Oversight and Government Reform, House of Representatives.
Why GAO Did This Study:
An accurate decennial census relies on finding and counting people”only
once”in the right place and collecting complete and correct
information. GAO reviewed the Department of Commerce‘s U.S. Census
Bureau‘s (Bureau) plans to (1) improve the census‘s accuracy”or
coverage”with its coverage follow-up operation, (2) assess the accuracy
of the census through its census coverage measurement program, and (3)
evaluate these efforts after 2010.
GAO reviewed documentation and interviewed Bureau officials about the
Bureau‘s coverage follow-up and measurement plans. GAO compared plans
for measuring coverage in 2000 and 2010 and interviewed 15 experts
about changes to the plans.
What GAO Found:
The Bureau plans to expand its coverage follow-up operation to improve
the accuracy of enumeration data collected in 2010. As in Census 2000,
coverage follow-up is intended to resolve count discrepancies (when the
number of household members reported on a completed questionnaire fails
to match the number of persons for whom information is collected) and
complete the enumeration of large households (more than six persons
living in a housing unit). For 2010, the Bureau researched new
techniques to increase the number of ways a person or household could
be selected for coverage follow-up. These include (1) using coverage
probes on the census form to identify cases with potential under- or
overcount problems, (2) using administrative records to identify
households that include persons who may not have been counted, and (3)
using computer matching to determine potential duplicate persons.
However, it is not clear on what basis the Bureau will select or
prioritize cases from this potentially expanded universe. The Bureau
plans to further assess the techniques during the 2008 Dress Rehearsal,
leaving little time to consider and implement improvements for the 2010
coverage follow-up operation. Because of the potential for the coverage
follow-up workload to increase, it will be important for the Bureau to
be clear on how it will decide which techniques to use and how it will
select coverage follow-up cases to ensure that the operation may best
improve coverage.
Although the Bureau‘s overall design for measurement is similar in many
respects to the design for 2000, its measurement plan differs in
important ways. The Bureau is planning to use a sample design similar
to that used in 2000 and still intends to produce net coverage error
estimates. However, the Bureau does not plan to use measurement results
to adjust the 2010 count of the nation‘s population and is planning to
make four significant changes: (1) conduct the measurement interview 4
months later than it did in 2000, (2) use a different method”logistic
regression”to estimate net error, (3) use estimates of the components
of coverage error in an effort to improve future decennials, and (4)
expand computer matching and clerical matching to match persons in its
postenumeration survey to the entire census. Still, the Bureau has not
specified when it will produce estimates of net coverage error and the
components of coverage error for the 2010 Census. The Bureau also has
not clearly described how it will associate the components of coverage
error to census operations in order to improve future decennials.
Because of the Bureau‘s changes to its coverage measurement program, it
is important for the Bureau to solidify its plans and share them with
stakeholders to ensure that there is an agreement on what the program
will produce and when the Bureau will produce its results.
The Bureau is in the early stages of planning how it will evaluate
coverage follow-up and measurement after the 2010 Census. As part of
this effort, the Bureau is considering recommendations of the National
Academy of Sciences for the Bureau‘s program of evaluations and
experiments for Census 2010. The Bureau has not yet developed a
timeline for completing its evaluation plans.
What GAO Recommends:
GAO recommends that the Secretary of Commerce direct the Bureau to
specify (1) criteria it will use to assess techniques for identifying
coverage follow-up cases; (2) when it will provide coverage measurement
estimates and how it will relate components of coverage error to
specific census operations to improve future decennials; and (3) key
decision points and plans for evaluating aspects of the 2010 coverage
follow-up and measurement efforts. In commenting on a draft of this
report, Commerce had no substantive disagreements with our
recommendations and will develop action plans for each one; and
provided technical comments that we incorporated as appropriate.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-414]. For more
information, contact Mathew J. Scirč at (202) 512-6806 or
sciremj@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Bureau Plans to Expand Coverage Follow-up to Improve Census 2010 but
Has Not Finalized Plans for Identifying Follow-up Cases:
Bureau Has Made Changes to Coverage Measurement for 2010 Intended to
Benefit the 2020 Census, but Key Aspects of Program Plan Remain
Unclear:
Bureau Plans for Evaluating the 2010 Coverage Follow-up Operation and
Census Coverage Measurement Program Are in Early Stages:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Comments from the Department of Commerce:
Appendix III: GAO Contact and Staff Acknowledgments:
Figures:
Figure 1: Decennial Census Population Net Undercount Rates from the
Bureau's Demographic Analysis in Percentages, 1940-2000:
Figure 2: Operation Flowchart for the Bureau's 2008 Dress Rehearsal
Coverage Follow-up Operation:
Figure 3: Undercount and Overcount Probes on Census Questionnaire for
the 2008 Dress Rehearsal:
Figure 4: Timeline for Coverage Follow-up Interviews and Census
Coverage Measurement Interviews Using Actual Dates for 2000 and
Estimated Dates for 2010:
United States Government Accountability Office:
Washington, DC 20548:
April 15, 2008:
The Honorable Henry A. Waxman:
Chairman:
Committee on Oversight and Government Reform:
House of Representatives:
Dear Mr. Chairman:
An accurate decennial census relies on finding and counting people--
only once--in the right place and collecting complete and correct
information. The Department of Commerce's U.S. Census Bureau (Bureau)
faces the daunting challenge of cost effectively counting a population
that is growing steadily larger, more diverse, and according to the
Bureau, increasingly difficult to find and reluctant to participate in
the census. These challenges are made even more difficult by the
Bureau's April 3, 2008 decision to redesign certain aspects of the
decennial census. Two basic errors can affect the count of the
population: omission of persons who should have been counted and
erroneous enumerations of persons who should not have been counted,
such as persons counted more than once or persons counted in the wrong
geographic place. To ensure accuracy, the Bureau, among other
operations, will conduct a coverage follow-up operation in 2010 to
determine if additional persons might have been missed (undercounted)
from already enumerated households, or if persons might have been
counted in error (overcounted). Further, apart from the enumeration
process, the Bureau will conduct a census coverage measurement program
to assess the accuracy of the 2010 Census.
The census is a critical national effort mandated by the Constitution.
Census data are used to apportion seats in the Congress, redraw
congressional districts, allocate billions of dollars in federal
assistance to state and local governments, and for numerous other
public and private sector purposes. The Bureau estimates that it has
generally undercounted the total population in prior censuses. For the
first time, in 2000, although estimates showed that it undercounted
certain population groups, the Bureau estimates that it somewhat
overcounted the total population. For 2010, the Bureau plans to improve
the accuracy of its population count by, among other things, expanding
its coverage follow-up operation. Further, the Bureau and others
questioned the accuracy of the data obtained from the Bureau's coverage
measurement program in 2000. As a result, for 2010, the Bureau plans to
revise its methodology for determining coverage error--the extent to
which persons are under-or overcounted in the enumeration process. As
agreed, we determined (1) how the Bureau plans to improve coverage
through its coverage follow-up operation; (2) how the Bureau's census
coverage measurement plans for assessing the accuracy of the 2010
Census compare with efforts in 2000 and what the potential impact will
be of those changes, including the Bureau's plans for using its
measurement program results to adjust population counts; and (3) how
the Bureau plans to evaluate the coverage follow-up and measurement
efforts after the 2010 Census.
To meet these objectives, we reviewed information about the Bureau's
coverage follow-up and coverage measurement plans and interviewed
Bureau officials. We reviewed relevant evaluations from Census 2000 and
studies conducted by the Bureau and others, including the National
Academy of Sciences. Further, we interviewed 15 experts about the
Bureau's coverage measurement methodology and plans. We identified
these experts based on published reports as well as recommendations
from the Bureau and the National Academy of Sciences. (For a list of
experts we interviewed, see app. I.) We conducted this performance
audit from January 2007 to March 2008 in accordance with generally
accepted government auditing standards. Those standards require that we
plan and perform the audit to obtain sufficient, appropriate evidence
to provide a reasonable basis for our findings and conclusions based on
our audit objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objectives.
Results in Brief:
The Bureau plans to improve coverage by expanding the techniques it
uses to identify and select cases for follow-up. However, it is not
clear on what basis the Bureau will decide which techniques to use to
identify potential cases, and which of these cases it will select or
prioritize for coverage follow-up. As in Census 2000, coverage follow-
up is intended to resolve count discrepancies (when the number of
household members reported on a completed questionnaire fails to match
the number of persons for whom information is collected) and complete
the enumeration of large households (more than six persons living in a
housing unit). The Bureau plans to include all such count discrepancy
cases and all large households in its coverage follow-up universe in
2010. To improve coverage for 2010, the Bureau researched new ways to
identify a more comprehensive set of cases for follow-up. Specifically,
to increase the number of ways a person or household could be selected
for follow-up, the Bureau tested three new coverage follow-up
techniques: (1) using coverage probes on the census form mailed to
households and during the nonresponse follow-up interviews to identify
instances where persons may be under-or overcounted; (2) using
administrative records to identify households that include persons who
may not have been counted; and (3) using computer matching to determine
potential duplicate persons. The Bureau is uncertain how many cases for
follow-up are likely to be found based on these new techniques.
Nonetheless, the number of follow-up cases could be very large and very
costly to resolve. Based on its experience with the 2008 Dress
Rehearsal, the Bureau expects to decide what techniques it will use for
identifying and selecting cases during the 2010 Census coverage follow-
up operation and how many cases it can afford to select for follow-up.
Bureau officials explained that they intend to consider cost and the
percentage of cases resulting in a change relative to the cost--known
as yield--but the Bureau has not fully specified how it will make
decisions about expanding the coverage follow-up universe or how it
will select or prioritize cases from those identified by the new
techniques. Using results from the 2006 Census Test, the Bureau has
already decided that coverage follow-up will only be conducted by
telephone rather than personal visits because the yield was higher for
telephone follow-up. This decision, however, does not necessarily
reflect trade-offs in the limitations of each technique used to select
cases for follow-up. For example, coverage may not be improved for
groups for which the Bureau does not have telephone numbers--the very
wealthy or the very poor. Given that the Bureau expects to use the
dress rehearsal to learn more about the expanded selection techniques,
and that the dress rehearsal has been delayed by a month, the Bureau
will have little time to complete its final plans for coverage follow-
up for the 2010 Census. Because of the potential for the coverage
follow-up workload to expand, it will be important for the Bureau to be
clear on how it will decide which techniques to use and cases to select
to ensure that coverage follow-up may best improve coverage.
Although the Bureau's plan for coverage measurement in 2010 is similar
in many ways to its measurement design for 2000 and previous coverage
evaluation programs, its plan differs in important ways, including its
overall objectives, timing, statistical techniques used, and use of
computer matching. Nonetheless, much of what the Bureau will do is yet
to be decided, including most notably, when the Bureau will produce
estimates of net coverage error and components of coverage error, and
how it will meet one of its objectives for coverage measurement in
2010--improving future census operations. One important aspect of the
Bureau's plans for 2010--specification of the logistic regression
model--cannot be known until after the Bureau collects the 2010
coverage measurement data. Similarities between 2000 and 2010 include
the Bureau's sample design for coverage measurement[Footnote 1] and
that the Bureau once again intends to use a dual systems approach to
produce net coverage error estimates--or the difference between the
coverage measurement program's estimate of the total population and the
decennial count of the total population.[Footnote 2] However, several
of the Bureau's objectives for the 2010 coverage measurement program
differ. For 2010, the Bureau places greater focus on improving future
decennials. In contrast, for 2000, the Bureau designed coverage
measurement to consider adjusting the count of the nation's population.
The Bureau does not plan to use coverage measurement results to adjust
the 2010 count of the nation's population because it believes that with
a sufficiently correct count, adjustment would introduce as much or
more error than it was designed to correct. The Bureau's measurement
program in 2010 differs in other significant ways from that in 2000,
for example, (1) starting the coverage measurement interview about 4
months later, to permit expansion of the Bureau's efforts to conduct
the coverage follow-up operation;[Footnote 3] (2) estimating net
coverage error using logistic regression, which is a standard
statistical modeling technique; (3) estimating the components of net
coverage error to better understand the details of erroneous
enumerations and omissions; and (4) expanding computer matching and
clerical matching to match persons in the coverage measurement
program's postenumeration survey to the entire census to resolve or
identify duplicates.[Footnote 4] Bureau officials said that the
coverage measurement interview will begin later because they expect to
expand the coverage follow-up operation, and that to maintain
independence and minimize possible contamination between enumeration
and measurement, the Bureau did not want to overlap coverage follow-up
and coverage measurement. Experts we interviewed were generally
concerned about the contamination issue but also viewed the time delay
as a very significant challenge because respondent recall[Footnote 5]
is vital to conducting the measurement program. The Bureau has
simulated the use of its new method of estimating net error--logistic
regression--by using data from 2000 and found it to work. The Bureau
will be unable to fully specify the model it will ultimately use until
after the data collection phase of its coverage measurement program.
One aspect of its plans that the Bureau should be able to specify now
is when it will produce net coverage error estimates and components of
coverage error estimates for 2010. Further, though the Bureau has
preliminary plans--using data table shells--to associate components of
coverage error to census operations, the Bureau has not clearly
explained how it plans to meet its objective of improving future
decennials. For the most part, experts we interviewed, after they
reviewed our summary of the Bureau's coverage measurement plans, stated
that the Bureau had improved its research model and had taken
significant and constructive steps in attempting to address key issues
for coverage error, but several experts noted that more detailed
planning is needed. Because of the Bureau's changes to its coverage
measurement program, it is important for the Bureau to make its plans
clear and share them with stakeholders to ensure that there is an
agreement on what the program will produce and when the Bureau will
produce its measurement results.
The Bureau's plans for evaluating its coverage follow-up operation and
census coverage measurement program for the 2010 Census are in early
stages of development. In terms of evaluation, the Bureau suggested
research on various aspects of coverage follow-up and measurement and
contracted with the National Academy of Sciences to recommend proposed
research for the 2010 Census. The academy's final report is due by
September 2009, but an interim report was issued December 2007. The
interim report contained recommendations for the Bureau's program of
evaluations and experiments for the 2010 Census. Bureau officials have
no timeline yet for making decisions about the evaluations of the
Bureau's coverage follow-up operation or census coverage measurement
program following the 2010 Census, and they noted that to determine
which recommendations to implement, the Bureau will consider costs and
staffing needs.
We are recommending that the Secretary of Commerce direct the Bureau to
provide Congress and other census stakeholders with more specific plans
on conducting coverage follow-up and census coverage measurement and
evaluating the results of those programs for the 2010 Census. To
improve program operations, we recommend that the Bureau specify the
criteria it will use to assess the techniques for identifying cases for
coverage follow-up in the 2008 Dress Rehearsal and the timeline it
plans to follow that will ensure that the results of the assessment may
be used in planning for 2010. To enhance transparency and oversight for
improving coverage measurement for the 2010 Census, we recommend that
the Bureau describe when it will provide estimates of net coverage
error and components of coverage error and how it plans to relate
components of coverage error to census operations in order to improve
future decennials. Finally, we recommend that the Bureau provide key
decision points and plans for evaluating aspects of the 2010 Census
coverage follow-up operation and census coverage measurement program.
The Secretary of Commerce provided written comments on a draft of this
report. The comments are reprinted in their entirety in appendix II and
we have incorporated in the report, as appropriate, the Department of
Commerce's technical comments. Commerce had no substantive
disagreements with our recommendations and stated that it will develop
formal action plans for each recommendation. While Commerce believes a
timeline for producing estimates of net coverage error and components
of coverage error is certainly a reasonable expectation, Commerce noted
that it is unclear as to how it can at this time provide any specifics
on how coverage results might improve future censuses before the
coverage data are obtained and analyzed. We recognize that the Bureau
has developed table shells that associate components of coverage error
to census operations, but these table shells and plans for how the
Bureau will conduct its analyses have not been shared with the Congress
and other stakeholders. In addition, it is not clear from current
Bureau plans how it will go about obtaining the data required to
complete the tables.
Background:
Historically, the census has been affected by undercounts--those who
were missed in the decennial count. Further, certain groups may be
undercounted more than others--such as babies, minorities, and renters
who move often. Seeking to obtain an accurate count has been a concern
since the first census in 1790. Concern about undercounting the
population continued through the decades. In the 1940s, demographers
began to obtain a more thorough understanding of the scope and nature
of the undercount. For example, the selective service registration of
October 1940 showed 2.8 percent more men than the census count.
According to the Bureau, operations and programs designed to improve
coverage have resulted in the total undercount declining in all but one
decade since the 1940s. These measures of coverage are based on
demographic analysis, which compares the census count to birth and
death certificates and other administrative data (see fig. 1).
Figure 1: Decennial Census Population Net Undercount Rates from the
Bureau's Demographic Analysis in Percentages, 1940-2000:
This figure is a versicle bar graph showing the decennial census
population net undercount rates from the bureau's demographic analysis
in percentages, 1940-2000. The X axis is the year, and the Y axis is
the percentage.
Year: 1940;
Percentage: 5.4%.
Year: 1950;
Percentage: 4.1%.
Year: 1960;
Percentage: 3.1%.
Year: 1970;
Percentage: 2.7%.
Year: 1980;
Percentage: 1.2%.
Year: 1990;
Percentage: 1.6%.
Year: 2000;
Percentage: 0.1%.
[See PDF for image]
Source: U.S. Census Bureau.
[End of figure]
In contrast to the long-standing use of demographic analysis to
estimate census undercount, modern coverage measurement began with the
1980 Census, when the Bureau compared decennial figures to the results
of an independent sample survey of the population. In using statistical
methods such as these, the Bureau began to generate detailed measures
of the differences among undercounts of particular ethnic, racial, and
other groups. In 1990, the Bureau relied on its Post-Enumeration Survey
to verify the data it collected through the 1990 Census. For this
effort, the Bureau interviewed a sample of households several months
after the 1990 Census, and compared the results to census
questionnaires to determine if each sampled person was correctly
counted, missed, or double counted in the census. Using the definition
of those missed minus those double counted, the Bureau estimated
whether there was a net undercount or net overcount. For the 1990
Census, the Bureau estimated a net undercount of about 4 million
people. During Census 2000, for the first time, the Bureau's coverage
measurement program measured a net population overcount. To estimate
the accuracy of the 2000 Census, the Bureau conducted its Accuracy and
Coverage Evaluation, expecting it could be used to adjust the Census
2000 results for all nonapportionment purposes if it improved the
census data. The original March 2001 estimates of the Accuracy and
Coverage Evaluation, which was an independent sample survey designed to
estimate the number of people who were under-and overcounted in the
census, a problem that the Bureau refers to as coverage error, became
available in time to correct census files. However, in a revised March
2003 evaluation, the Bureau was unable to conclude that the adjusted
data were more accurate and determined that the 2000 Census tabulations
would not be adjusted for any purpose. The decision not to adjust was
consistent with decisions made during the 1990 decennial, when the
Department of Commerce decided, because of other problems, not to use
the postenumeration survey to adjust the decennial count.
During this decade, the Bureau has conducted a testing and evaluation
program to assess certain activities that it believes can improve the
accuracy of the census. In preparing for the 2010 Census, the Bureau
has conducted two field tests--in 2004, the Bureau tested in the Queens
Borough of New York City and southwest Georgia and, in 2006, tested at
the Cheyenne River Reservation in South Dakota and parts of Austin,
Texas, in Travis County. In 2003 and 2005, the Bureau also conducted
national surveys to test various aspects of changes planned for 2010.
We reported earlier about the importance of these earlier assessments
leading to a design that would be sufficiently mature so that the dress
rehearsal for the 2010 Census demonstrates the feasibility of the
various operations and technologies planned for the decennial under
conditions that are as close as possible to the actual census.[Footnote
6] The Bureau is currently conducting its dress rehearsal, one of the
final planning activities for the decennial. Census Day for the 2008
Dress Rehearsal activities in sites located in North Carolina and
California will be May 1, 2008, a delay of 1 month from the Bureau's
initial plan.
The first 2010 Census operation, building an accurate address list,
overlaps with dress rehearsal activities and has also already
begun.[Footnote 7] For the 2010 Census, the majority of the housing
units in the country will receive paper census questionnaires delivered
either by mail or by census field workers before April 1, 2010. Those
households that do not return their questionnaires will be contacted by
census field workers during the nonresponse follow-up operation to
determine the number of people living in the housing unit on Census Day
and other information. The Bureau also uses other techniques for
counting persons. For example, people in group quarters (such as
college dormitories, homeless shelters, and nursing homes) will be
enumerated in separate operations at the facilities by enumerators who
will then leave forms listing the names of the people living or staying
there and then return to pick up the completed forms and, if necessary,
conduct follow-up interviews.
For the 2010 Census, the Bureau has tested and planned revisions to the
census form with the goal of improving the census' coverage. The
revisions are intended to improve methods for ensuring that persons are
counted in the correct place and only once.[Footnote 8] These changes
include revising residency rules, simplifying the instructions provided
to respondents on residency, and including questions--called coverage
probes--about persons who may be missing or incorrectly included in the
response. For Census 2000, the Bureau relied on 31 residence rules to
instruct respondents on who to include on the mail-back form. The
Bureau revised these 31 rules to 1 rule with 13 broad living situations
for 2010. For 2010, the Bureau is also planning on using two coverage
probes, one asking about potentially missed people (undercounted) and
one asking about potentially duplicated people (overcounted), on the
census form. Although the Bureau used coverage probes in the
nonresponse follow-up operation for 2000, it did not include probes in
the 2000 Census mailed questionnaires as is now planned for 2010. The
Bureau intends to use answers to these coverage probes and other
techniques to determine if an enumeration is likely to be incorrect or
incomplete and, therefore, if cases should be selected for follow-up.
The Bureau uses the results of the coverage follow-up operation to
correct the results of the initial enumeration. Coverage follow-up
interviews with respondents determine if changes should be made to
their household rosters as reported on their initial census returns.
Specifically, these results lead to a correction of census information
and counts, as coverage follow-up results are combined with the initial
enumeration and are used as the official enumeration of the household.
Bureau Plans to Expand Coverage Follow-up to Improve Census 2010 but
Has Not Finalized Plans for Identifying Follow-up Cases:
For Census 2010, the Bureau is planning to improve coverage by
expanding its coverage follow-up operation to obtain more accurate
enumeration data. To prepare for this, the Bureau has conducted tests
of an expanded follow-up operation. As in Census 2000, coverage follow-
up is intended to resolve count discrepancies and complete the
enumeration of large households. For 2010, the Bureau also plans to use
additional techniques to increase the number of ways a person or
household could be selected for follow-up. However, the Bureau has not
finalized plans for how to identify and select cases from the expanded
follow-up universe. The expansion of selection criteria for coverage
follow-up has the potential to dramatically increase the workload for
this operation and, as a result, increase costs. While the Bureau plans
to budget for coverage follow-up in order to achieve the best results
or highest yield of changes to household rosters, it is unclear on what
basis the Bureau will assess the additional techniques for identifying
potential cases or the basis for selecting cases for follow-up using
the expanded techniques. The Bureau plans to assess its experiences
with coverage follow-up, including important interfaces within the
Bureau and between the Bureau and its contractor, during the 2008 Dress
Rehearsal in making further decisions on follow-up for 2010. For
example, three separate groups within the Bureau have responsibility
for selecting follow-up cases, but one of those groups eliminates
overlap in cases before sending them to the contractor for telephone
follow-up. Recent decisions to delay the dress rehearsal, including
follow-up activities, increase the risk that the results of additional
testing of these techniques in 2008 will not be sufficiently timely or
complete to improve operations for 2010, and the Bureau has not stated
when it will complete its assessment of coverage follow-up for the 2008
Dress Rehearsal.
Bureau Plans to Expand Techniques for Identifying Coverage Follow-up
Cases:
The Bureau is planning to improve coverage by expanding its coverage
follow-up operation. To meet this objective, the Bureau expects to rely
on previously used techniques as well as techniques it is now
developing to identify cases for follow-up. As in Census 2000, coverage
follow-up for 2010 will include:
* resolving count discrepancies, when the number of household members
reported on a completed questionnaire fails to match the number of
persons for whom information is collected, and:
* completing the enumeration of large households, in which there are
more than six persons living in a housing unit.
For 2010, the Bureau also plans to use additional techniques for
selecting a person or household for follow-up. The techniques planned
include:
* using questions--called coverage probes--on the census form itself
and in the nonresponse follow-up interviews to identify where persons
may be omitted or counted more than once,
* using administrative records to identify persons who may be omitted,
and:
* using computer matching from the universe of census returns to
identify persons who may have been counted more than once.
Although the Bureau has not finalized coverage follow-up plans for the
2010 Census, in June 2007, the Bureau developed a work flow and
narrative for the 2008 Dress Rehearsal coverage follow-up operation.
This work flow shows the process for collecting enumeration data,
selecting coverage follow-up cases, and conducting follow-up (see fig.
2). Once the Bureau selects a case for follow-up, it plans to work
through its Decennial Response Integration System (DRIS) contractor,
which will conduct coverage follow-up interviews by telephone. The
telephone interviews determine if changes should be made to the
household rosters as reported on the initial census returns. The
questions in the follow-up interview are designed to identify if
persons were omitted or counted in error because they should have been
counted at a different address or not at all--for example, because a
person had been born after or died before Census Day.
Figure 2: Operation Flowchart for the Bureau's 2008 Dress Rehearsal
Coverage Follow-up Operation:
This figure is a an operation flowchart showing the bureau's 2008 dress
rehearsal coverage follow-up operation.
[See PDF for image]
Source: GAO analysis of U.S. Census Bureau data.
[End of figure]
As shown in figure 2, enumeration data will be captured by DRIS. Also,
in selecting cases for follow-up, the Bureau will identify any overlap
in cases represented by large households, count discrepancies, coverage
probes, administrative records review, and computer matching. While the
Bureau has tested new techniques for coverage follow-up, it has not yet
tested these steps with the new processing interfaces within the
Bureau--Decennial Systems and Processing Office, Decennial Statistical
Studies Division, and Data Integration Division--and between the Bureau
and DRIS. During the 2008 Dress Rehearsal, from May 27 to August 9,
2008, the Bureau will not only be assessing various aspects of the
expanded follow-up, but it will also be testing for the first time the
DRIS integration of data as well as the Bureau's process for
identifying overlap cases. After the Bureau selects cases for follow-
up, the DRIS contractor assigns an identification number to each case
and oversees telephone interviews. Each case may receive up to seven
follow-up telephone calls, and the DRIS contract includes such aspects
as on-demand automated call distribution, call routing, reporting, and
quality assurance monitoring.
Resolving Count Discrepancies:
For 2010, as in Census 2000, all count discrepancy cases identified on
mailed paper census forms will be part of the coverage follow-up
operation. The Bureau will check completed questionnaires for
discrepancies between the number of persons reported as members of a
household and the number of persons for whom census information was
provided on the census forms. For example, questionnaires that fail
edit checks might include cases where the respondent may indicate that
five people were living in the housing unit on Census Day, but there is
information for only two people.
Enumerating Large Households:
As in 2000, the Bureau is also aiming to improve coverage of people in
housing units by completing the enumeration of large households,
meaning those that indicate on their census forms that there are more
than six persons in the housing unit and for which there was not enough
room on the mailed questionnaire form for self-response. The
questionnaire has space for only six household members to be fully
listed. The nonresponse follow-up interview is not limited in the
number of people that can be reported and all household members can be
enumerated during nonresponse follow-up; therefore, no large household
cases will need to be enumerated during the coverage follow-up
operation as a result of the nonresponse follow-up interview.
Using Coverage Probes to Identify Potential Under-or Overcounting:
The Bureau plans to expand its universe for coverage follow-up in 2010
by including cases resulting from the use of coverage probes. For the
dress rehearsal, the Bureau plans to contact anyone who affirmatively
answers the undercount question (which is at the household level) and
the overcount question (which is at the person level) (see fig. 3). If
either or both coverage probes prove unproductive in the dress
rehearsal, Bureau officials indicated that they might be eliminated
from the follow-up operation. The coverage probes are included on the
mail-back questionnaire and among the questions asked during the
nonresponse follow-up. During nonresponse follow-up, for example, if a
person responds in the affirmative to the coverage undercount probe,
the interviewer collects the name and demographic information for the
added person. All names of individuals added to the household
roster[Footnote 9] through the undercount probe during nonresponse
follow-up will be sent to the coverage follow-up operation for
additional follow-up.
Figure 3: Undercount and Overcount Probes on Census Questionnaire for
the 2008 Dress Rehearsal:
This figure is a copy of undercount and overcount probes on census
questionnaire for the 2008 dress rehearsal.
[See PDF for image]
Source: GAO presentation of U.S. Census Bureau information.
[End of figure]
As shown in figure 3, for the 2008 Dress Rehearsal, the undercount
probe asks if there are any additional people staying at the residence
on Census Day that were not included earlier--children, such as newborn
babies or foster children; relatives, such as adult children, cousins,
or in-laws; nonrelatives, such as roommates or live-in baby sitters; or
people staying at the residence temporarily. The overcount probe for
the 2008 Dress Rehearsal asks if persons sometimes lived or stayed
elsewhere--such as while attending college, while in the military,
while staying at a seasonal or second residence, during a child custody
arrangement, while in jail or prison, while in a nursing home, or for
another reason. The overcount probe is asked of the first six household
members on the mail-back questionnaire and of all persons in the
nonresponse follow-up interview.
Evaluations of the coverage follow-up operation for the Bureau's 2004,
2005, and 2006 tests examined the performance of both the undercount
and overcount probes. In the Bureau's 2004 test, the overcount probe
was found to be more effective in finding households with erroneous
enumerations than the undercount probe was in finding persons omitted
from the count. In this test several problems were identified with the
undercount probe, most notably a low rate of added residents to the
household roster. Results from the test showed that of the 3,919
respondents at the Georgia test site who answered the undercount probe
affirmatively, only 53 persons were added to the household roster. The
Bureau noted that the low rate of added persons was "perplexing." Other
problems included difficulties with the interviewer reading the probe
as worded and indications that use of the probe may have actually
increased the number of count discrepancy cases.
In 2005, the Bureau tested two different versions of both the
undercount and overcount probes, and changes to both probes were
adopted for the 2006 Census Test. During the 2006 test, the Bureau
found that the overcount probe "performed as expected" and generated a
higher rate of deleted persons from the census test roster for both the
mail-back questionnaire and nonresponse follow-up operation than other
new techniques tested to identify coverage follow-up cases. Although
the Bureau designed the undercount probe to add persons, the undercount
probe resulted in more deleted persons than added persons.[Footnote 10]
The Bureau noted that over half of the deletions stemmed from cases in
which the respondent either answered "yes" to both coverage probes or
the respondent answered "yes" to the undercount probe and had a count
discrepancy. Indeed, in a separate analysis that examined the rate of
overlap among coverage follow-up cases--meaning households were flagged
for coverage follow-up through more than one technique--the Bureau
found that the highest significant rate of overlap was among cases
where the respondent answered "yes" to both coverage probes.
Additionally, there were some operational difficulties using the
undercount probe during the nonresponse follow-up operation in the 2006
Census Test.[Footnote 11] For example, the Bureau found that although
persons answered the undercount probe affirmatively (1,234 households)
during nonresponse follow-up, very few persons (8) were added to the
coverage follow-up roster.[Footnote 12] The Bureau referred to this
number as "surprisingly low," because if a household replies "yes" to
the undercount probe, the Bureau expected to see a person added to the
roster for coverage follow-up. In its evaluation of coverage follow-up
during the 2006 Census Test, the Bureau speculated that there could
have been operational error during nonresponse follow-up that led to
this discrepancy. Furthermore, of those 8 persons who were added and
appeared in coverage follow-up, only 3 could be confirmed as residents
during follow-up, leading the Bureau to conclude that the undercount
question added persons incorrectly more than half of the time after
nonresponse follow-up. Although the Bureau has made some slight
modifications to the response options for the overcount probe it plans
to use in the dress rehearsal, it plans to use the same undercount
probe for the dress rehearsal as it did for the 2006 test.
Using Administrative Records to Identify Potentially Omitted Persons:
For the dress rehearsal, the Bureau plans one wave of matching with
administrative records to identify duplicate or suspected incorrect
data. For administrative matching, the Bureau will use the Enhanced
Statistical Administrative Records System, which is a Bureau-wide
resource that collects certain data from the Internal Revenue Service,
the Social Security Administration,[Footnote 13] the Selective Service,
the Department of Housing and Urban Development, as well as other
federal agencies. The Bureau will match census returns against records
from these sources using various algorithms to associate persons with
housing units to identify households with potential
undercounting.[Footnote 14] The Bureau plans to match names on census
questionnaires with names in the administrative database--if a
household is found to have more persons in the administrative records
than on the census form, it would be a candidate for coverage follow-
up. The Bureau first tested selecting cases for follow-up through
comparison with administrative records in 2005. For the 2006 Census
Test, all of the cases with matches from the administrative records
search had personal visits; however, Bureau officials noted that
personal visits were conducted because staff had sufficient time, given
the smaller geographic area of a test site. Personal visits will not be
part of the national 2010 coverage follow-up operation. The 2008 Dress
Rehearsal marks the first time that cases selected through
administrative record matching will be tested in the coverage follow-up
telephone operation.
Using Computer Matching to Identify Persons Who May Have Been
Duplicated:
For the dress rehearsal, the Bureau will conduct several waves of
computer matching for its unduplication process. In 2000, the Bureau
had difficulty with duplicate enumerations and did not have a planned
operation for dealing with the problem; officials explained that in
2000 there was an "ad hoc" plan that they developed "on the fly" for
identifying duplicates. However, Bureau officials noted that one of the
lessons learned from 2000 was that the Bureau needed to expand the
search for duplicates by computer matching. Thus, to improve the 2010
Census, along with the overcount probe specifically designed to try and
find duplicates, the Bureau will also match census returns against the
universe of all other census returns to identify duplicates nationwide;
for the dress rehearsal, the matching will be conducted within the test
site, rather than on a national scale. While the Bureau has been
evaluating and testing this procedure, the limited likelihood of
duplication of individuals that will occur within census test sites of
limited geographic size, as compared to the actual national decennial,
limits the Bureau's ability to forecast how the nationwide matching and
unduplication system will affect workload for coverage follow-
up.[Footnote 15]
Although the Bureau has not finalized its coverage follow-up
techniques, it plans to apply a computer-based record linkage system to
the data collection files during the 2010 Census to match files against
themselves to find potential duplicates based on housing unit and
person information. Those households would then be selected for follow-
up. For example, erroneous enumerations are created when duplicated
people are counted correctly in one housing unit and incorrectly in
another housing unit. Duplicated housing units, which occur when
multiple addresses of the same housing unit are identified during
census operations, are removed. Further, in contrast to Census 2000,
the addresses that could be included in the census in both the housing
unit and group quarter enumerations can no longer cause duplication
because, according to the Department of Commerce, the Bureau now has
one list for both housing units and group quarters. Consequently,
according to Bureau officials, the Bureau faces the challenge of
identifying the types of duplicates that can be most easily identified
and addressed, which will result in the most cost-effective yield.
The Bureau's 2006 test of computer matching found that potential
duplicates generally were not deleted. Specifically, in its evaluation
of computer matching during the 2006 Census Test, the Bureau found that
93 percent of respondents marked as possible duplicates of persons
counted outside the census block were neither deleted nor linked to a
deleted match. The Bureau considered this result disappointing. It
noted that the coverage follow-up interviews did not resolve the cases
very well because follow-up interviews were not completed for 24
percent of the cases and, in 61 percent of the cases, respondents did
not confirm during the follow-up interview that they lived or stayed
somewhere else. The interviewer cannot provide information about the
suspected duplicate during the follow-up interview because of concerns
related to confidentiality; the interview is limited in its potential
for success in that the respondent has to volunteer information about
the suspected duplicate case. Consequently, the Bureau is not only
challenged in identifying suspected duplicate cases, but in actually
resolving the cases as well.
Bureau officials explain that for the unduplication process, the Bureau
will assign a match score and identify links among suspected
duplicates. The Bureau plans to develop a match score threshold that
will vary by level of geographic distance between matches. The Bureau
will then assess the quality of the match based on the match score and
conduct further research on the cases exceeding this threshold. For
example, once the Bureau identifies potential duplicates, it determines
whether the potential duplication is a potential housing-level problem
(duplicated persons identified within a census block) or person-level
problem (more long distance-identified duplicates nationally). If it is
a housing-level problem, then the Bureau would consider using personal
visits by field staff from another of its operations, the field
verification operation, to verify housing unit additions. Further,
according to the Department of Commerce, field verification may be
undertaken to determine if housing units should be deleted from the
list. If it is a person-level problem, the Bureau would consider a
coverage follow-up operation telephone interview to determine if the
person is counted twice.
Bureau Will Rely on Assessment of Dress Rehearsal to Complete Coverage
Follow-up Plans for the 2010 Census:
The Bureau is developing a number of techniques for selecting cases for
follow-up and plans to expand the coverage follow-up operation overall.
However, it has not determined which techniques it will use, what
thresholds it will use in selecting cases for follow-up, or the
resources it expects to devote to the coverage follow-up operation. The
most pressing question remaining is how the Bureau will prioritize its
expanded universe of follow-up cases identified through coverage
probes, administrative record matching, and the unduplication process.
Currently, households that display count discrepancies, are large,
reply in the affirmative to coverage probes, are flagged through
comparisons with administrative records, or are flagged as potential
duplicates are all eligible for follow-up. Although there may be a
large number of eligible cases for coverage follow-up, the Bureau is
unsure how many cases it can financially afford for follow-up, and
discussions are still under way within the Bureau about what units will
be sent to the coverage follow-up operation. The Bureau is reviewing
data from tests--such as the add and delete rates within each type of
case--to aid in determining what types of techniques will be most
productive in terms of identifying overcounted and missed persons.
The Bureau plans to rely on additional data from its assessment of the
coverage follow-up operation in the 2008 Dress Rehearsal in making
decisions about which techniques, including coverage probes,
administrative record matching, and the unduplication process, it plans
to use for identifying and selecting cases for coverage follow-up in
2010. Dress rehearsal plans for coverage follow-up should mirror the
plans intended for the decennial because the 2008 Dress Rehearsal will
be the Bureau's last opportunity to assess the various procedures and
systems for 2010 under as close to census-like conditions as possible.
However, the expanded coverage follow-up operation conducted during the
dress rehearsal will not necessarily be what the Bureau implements in
2010 because the Bureau plans to use the dress rehearsal to assess key
aspects of a likely coverage follow-up operation, including (1) the
potential coverage follow-up workload and how it may be distributed,
(2) the extent to which the coverage follow-up operation improves
census coverage through such factors as the percentage of added
residents and deleted residents, and (3) problems administering the
follow-up interview and managing work. The assessment will also address
problems with the operational work flow and interfaces and provide
recommendations to fix or improve these problems. While the Bureau's
expectation is that it will have the necessary data from the dress
rehearsal to inform the final plans for the 2010 coverage follow-up
operation, Bureau officials have noted that the delayed dress rehearsal
(a delay in the Census Day from April 1 to May 1, 2008) may make it
more difficult to incorporate the findings from the dress rehearsal
into the Bureau's plans for the 2010 Census.
Bureau officials have stated in several forums that resource
constraints are a factor in the final decision on the scope of follow-
up in 2010. The expansion of selection criteria has the potential to
dramatically increase the workload for coverage follow-up in 2010 and,
as a result, increase costs. Bureau officials have cautioned that it is
not necessarily cost-effective for the Bureau to follow up on the
entire universe eligible to be included in the coverage follow-up
operation. Thus, the Bureau will be using results--such as the
percentage of cases resulting in a change relative to the cost or
yield--from the dress rehearsal to decide which techniques to use and
how many cases it can afford to include for follow-up.
Evaluations of the impact of the criteria the Bureau will use in
judging what techniques to use for identifying cases for follow-up or
the thresholds for selecting specific cases for follow-up will be
critical to determining the final plan for the coverage follow-up
operation. The Bureau has made one judgment about how it will proceed
for 2010 as a result of its 2006 test. Specifically, in its evaluation
of the 2006 Census Test, the Bureau calculated that the cost was $155
per roster change for coverage follow-up cases done by telephone versus
$766 per roster change for cases done by personal visits. Consequently,
the Bureau concluded that it should conduct follow-up interviews by
telephone. This decision, based partly on the percentage of cases with
a change and partly on cost, represents two criteria.
Effective program management, including assessment of risks,[Footnote
16] requires that agencies identify information that is critical to
achieving important program objectives.[Footnote 17] In this case, the
Bureau should clearly state the basis for deciding which techniques to
use or the number of cases it will include for coverage follow-up.
Because the Bureau has not specified these criteria, it is not clear
what criteria--other than roster change and cost--the Bureau will
consider. For example, the Bureau noted in its decision to conduct
coverage follow-up by telephone that not adding a personal visit
component to the operation would limit its ability to reduce coverage
errors for households it could not reach by telephone. Bureau research
indicated that households for which it did not have telephone numbers
tended to be very wealthy or very poor. Thus, coverage may not be
improved for these demographic groups through telephone follow-ups
alone.
Bureau Has Made Changes to Coverage Measurement for 2010 Intended to
Benefit the 2020 Census, but Key Aspects of Program Plan Remain
Unclear:
The Bureau's overall design of its census coverage measurement program
for the 2010 Census is similar in many respects to its measurement
design for 2000 but differs in some important ways. The objectives,
timing, and some techniques to be used for 2010 differ from those used
in 2000; however, the Bureau has not made clear when it will produce
estimates of net coverage error and components of coverage error, nor
does it make clear how it will meet one of its objectives for coverage
measurement in 2010--improving future census operations. In 2000, the
Bureau focused on estimating net coverage error with the possibility of
producing adjusted numbers at the census block level. In contrast, for
2010, while still intending to produce net coverage error estimates at
higher levels of geography--the national and state levels--the Bureau
has refocused its efforts in order to use its measurement program to
help improve enumeration methodologies for the future. Importantly, the
Bureau does not plan for the measurement program to prepare adjusted
numbers for the 2010 Census. In addition to starting coverage
measurement interviews 4 months later than it did in 2000, the Bureau
has also made several other important changes to its measurement
program. For example, to better estimate net coverage error, the Bureau
plans to use logistic regression for the first time. The Bureau also
plans to assess the association of components of coverage error to
various census operations. Finally, to better measure coverage, the
Bureau plans to better identify duplicates by expanding its computer
and clerical matching. Expert statisticians and methodologists we
interviewed generally agreed that the Bureau's changes for its 2010
measurement program are an improvement from the 2000 coverage
measurement program; however, several experts noted that more detailed
planning is needed.
For the 2010 coverage measurement program, the Bureau expects to
include multiple operations. First, the Bureau selects its sample of
census block clusters to be included in the coverage measurement
program. Using the sample, the Bureau collects information on the
housing units in the census blocks to develop an address list
independent from that used in the census enumeration. The Bureau then
uses this address list to draw a sample of housing units to include in
the person interview operation. During this operation, Bureau field
staff interview--in person--residents of these housing units and
collect demographic, Interview Day and Census Day residency information
of all persons living at the housing unit. According to the Department
of Commerce, additional information will be collected in 2010 to
identify alternate addresses where persons on the coverage sample
roster may have been counted on Census Day and this information will be
used in person-based computer and clerical matching operations. Next,
the Bureau compares data collected during the person interviews with
all census enumerations. Where the Bureau is unable to make a match,
the Bureau follows up by sending interviewers back to the housing unit
to reinterview the person or persons in question. During the final
operation, the Bureau produces estimates of coverage error for persons
and housing units. For 2010, the Bureau plans to provide estimates of
net coverage error at the national level by race/ethnicity, age/sex,
and tenure--the status of an occupied housing unit as either owner
occupied or renter occupied. The Bureau will also produce estimates of
net coverage error for states.
2010 Sample Design for Census Coverage Measurement Is Similar to That
for 2000:
The Bureau's 2010 coverage measurement program sample design is similar
to that used for measurement in 2000. As in 2000, the Bureau plans to
use a 300,000 national housing unit sample. The methods are basically
consistent with the 2000 coverage measurement program and are expected
to yield similar levels of reliability, although slightly different
methods are being used to determine state sampling allocation. The
objective for 2010 is to have the sample allocated proportionally
across states and the District of Columbia, with a minimum of 1,800
housing units in each state. Moreover, the Bureau is using a
differential within state sample allocation in order to allow for
oversampling in areas as needed. Notably, according to Bureau
officials, it is possible that that the sampling numbers could still
change depending on budget or operational constraints. Most experts we
spoke with thought that the Bureau's sample design was appropriate for
its objectives.
Census Coverage Measurement Interviews Will Begin Later, and Estimates
Will Take Longer to Complete in 2010:
In 2000, the timeline for coverage measurement overlapped with the
Bureau's coverage follow-up efforts, though it was conducted
independently from the census enumeration; however, measurement and
coverage follow-up are planned to be conducted sequentially for the
2010 Census. Specifically, in 2010, the census coverage measurement
interview will begin after the coverage follow-up operation is
completed (see fig. 4). According to the Bureau's 2010 Census
Operations and Systems Plan, dated August 31, 2007, the Bureau expects
to conduct the person interviews for coverage measurement from August
20 through October 16, 2010. In 2000, the Bureau conducted the coverage
measurement person interviews from April 24 through September 11, 2000.
Figure 4: Timeline for Coverage Follow-up Interviews and Census
Coverage Measurement Interviews Using Actual Dates for 2000 and
Estimated Dates for 2010:
This figure is a timeline for coverage follow-up interviews and census
coverage measurement interviews using actual dates for 2000 and
estimated dates for 2010.
[See PDF for image]
Source: GAO analysis of U.S. Census Bureau data.
[End of figure]
According to the Bureau, the expansion of the coverage follow-up
operation contributed to delaying the measurement program in order to
maintain data independence and reduce the likelihood of data
contamination resulting from the overlap of follow-up and measurement
field operations.[Footnote 18] In past coverage measurement programs,
the person interview phase overlapped with coverage follow-up conducted
as part of the census enumeration. In 2010, the Bureau plans to delay
the start of the person interview phase for coverage measurement to
minimize the risk of data contamination. While many experts agreed that
delaying the coverage measurement interviews will reduce the risk of
data contamination, some experts also highlighted that the delay may
also negatively affect the data quality because it could contribute to
recall bias. Recall bias, as it pertains to coverage measurement, means
that a person incorrectly recalls where he or she or other household
members resided on Census Day and thus biases the coverage measurement
data. Two experts also noted that delaying coverage measurement
interviews could also affect quality of data on movers. Generally,
experts said that decreasing the amount of time between the coverage
follow-up interview and the coverage measurement interview would
benefit the coverage measurement data. However, a few experts who were
also familiar with the issue stated that they were unsure of the
magnitude of contamination and, therefore, did not know if the Bureau
should decrease the time. Several of these experts argued that the
Bureau could make a better case by using results from its contamination
studies to determine if the delay is warranted. In 2006, the Bureau
conducted a contamination study using the results of the 2006 Census
Test to determine if the coverage measurement person interviews caused
a difference in responses to the coverage follow-up
interviews.[Footnote 19] Although the Bureau stated that the number of
sample cases involved in the study was small and that the result should
be viewed with caution, it concluded that there might be some small
impact on coverage follow-up results if the measurement interview is
conducted at the same time as the coverage follow-up interview.
For 2010, the Bureau also expects to complete its coverage error
estimates later than was the case for the Bureau's initial 2000
estimates. In 2000, the Bureau produced initial estimates in March
2001; however, after evaluating the 2001 estimates, the Bureau released
revised estimates in March 2003. According to Bureau officials, the
Bureau has preliminary plans to complete coverage measurement
estimation[Footnote 20] for the 2010 Census sometime after October
2011[Footnote 21]--6 months after the Bureau provides population counts
for redistricting. Bureau officials have stated that the decision to
delay measurement estimation will allow for more time to adequately
evaluate the 2010 measurement data and estimates, noting that the
Bureau did not have sufficient time for analysis in 2000.
Objectives for the 2010 Census Coverage Measurement Differ from Those
for 2000, but How the Bureau Will Meet New Objectives Is Unclear:
The Bureau's 2010 coverage measurement objectives have changed from
those of Census 2000. In 2000, the Bureau focused on estimating net
coverage error with the possibility of adjusting the census count at
the census block level of geography. Though the Bureau still intends to
produce net error estimates for 2010--at the state level and higher--it
has refocused the coverage measurement program on improving future
enumeration methodologies; consequently, it does not plan to use the
results of its coverage measurement program to improve or adjust the
count of the nation's population in 2010. For 2010, the Bureau's goals
for measuring coverage accuracy include the following aspects:
* The Bureau plans to use a dual systems approach to estimate net
coverage error by using logistic regression instead of the
poststratification technique used in 2000. The Bureau believes its move
to logistic regression will also allow more flexibility because it can
incorporate a greater number and type of factors or predictors in the
model.
* The Bureau plans to improve future census operations by associating
estimates of components of coverage error, including erroneous
enumerations and omissions, to census operations.
* To better resolve and identify duplicate enumerations, a major source
of error in previous decennials and coverage measurement programs, the
Bureau plans to expand computer and clerical matching to match persons
in its postenumeration survey to the entire census from the more
limited matching done in 2000.
The Bureau plans to conduct a census coverage measurement program for
the 2010 Census, but Bureau officials said that they did not know when
the Bureau would produce detailed documentation describing its plans.
The Bureau has developed preliminary plans for associating components
of coverage error to census operations, but these plans also are not
finalized.
Bureau Does Not Plan to Adjust the 2010 Census in Time for
Redistricting:
According to the Bureau, it has no plans to use the results of its
coverage measurement program to improve or adjust the 2010 Census
count. In response to a May 12, 2003 letter from Representative William
Lacy Clay asking about decennial plans, the Bureau explained that with
a sufficiently correct count, adjustment would introduce as much or
more error than it was designed to correct. If the Bureau were asked by
the Congress to provide adjusted numbers, Bureau officials commented
that it would be unable to quickly or easily do so because the current
coverage measurement program's timeline produces estimates after
October 2011, and the Bureau has not designed the software to produce
estimates at lower geographic levels. Consequently, according to the
Bureau, it cannot prepare adjusted numbers in time to meet the legal
deadline for producing count data for redistricting, which is April 1,
2011. Furthermore, the Bureau is not planning to provide net error
estimates below the state level; such estimates are necessary for
redistricting.
Some of the interviewed experts agreed that producing adjusted numbers
would be challenging; however, several also said that the Bureau could
still provide some adjusted numbers if asked to do so. Regarding the
challenges, two experts stated that adjusted numbers could be produced
at a higher geographic level, the national level, but not accurately at
lower geographic levels, the state and local levels, without
appropriate changes to the current design and methodology. For example,
one expert explained that the Bureau would need to use
poststratification to produce estimates at the state and local levels.
Additionally, several experts noted that the 2010 coverage measurement
timeline poses a challenge for producing adjusted numbers by the April
1, 2011, redistricting deadline.
Bureau Plans to Use a Different Statistical Method for Estimating Net
Error but Has Not Fully Specified Its Plans:
The Bureau plans to use logistic regression to improve its estimation
of net coverage error. Of the experts familiar with logistic
regression, the majority thinks that the methodology is an improvement;
however, a few experts question how the Bureau will explain a more
complex method to its stakeholders. Previously, for the 2000 coverage
measurement, the Bureau used a poststratification[Footnote 22] method.
However, Bureau officials said that poststratification was limited
because of the small sample sizes of many of the strata. They explained
that logistic regression permits greater flexibility in dealing with
small sample sizes because it allows for a greater number and type of
factors or predictors in the model.[Footnote 23] By definition,
poststratification is limited because it only allows for a small number
of factors with defined categories to be included. In addition to
greater flexibility, logistic regression can potentially reduce errors
in estimation. According to Bureau officials, to verify estimates from
logistic regression, the Bureau is also considering using some
poststratification. Logistic regression provides greater flexibility in
dealing with small samples, but it may be more difficult
methodologically to use logistic regression to produce state-level
estimates by race/ethnicity, age/sex, and tenure or to estimate lower
than the state level. Some experts argued for continuing to produce
estimates using poststratification to enable direct comparisons of
error estimates to prior decennials and to verify that logistic
regression is working.
To enhance transparency and oversight for improving coverage
measurement for the 2010 Census, the Bureau should provide the Congress
with timely and accurate information to allow monitoring of agency
activities--specifically, timely and accurate information about its
plans for coverage measurement.[Footnote 24] The Bureau plans to refine
its estimation methods for 2010 by using logistic regression to
estimate net coverage error, but it does not have an operational or
working plan that outlines what resources--such as the amount of time
that it will take to run the data set(s)--will be used to complete this
operation. Further, despite conducting some exploratory logistic
regression work using the 2006 test data, as well as simulating its
estimation model for logistic regression using decennial data from 2000
and finding workable models, the Bureau has not documented what
approaches or processes it intends to apply in the further development
of the regression model or models it plans to use. In addition, the
Bureau does not have a written plan detailing how it will explain to
stakeholders and Bureau data users the rationale for moving from
poststratification to logistic regression and what effect this change
may have on data interpretation--particularly as it relates to
comparisons with past census data. Experts we interviewed also noted
some challenges, including communicating the new method to users of
census data and stakeholders, having an adequate number of staff to
complete the analysis, and designing the best models--such as choosing
the variables to include.
Bureau's New Focus on Estimating Components of Coverage Error Is
Intended to Benefit Future Decennials, but Plans Are Incomplete:
The Bureau plans to estimate the components of coverage error for 2010
in order to associate the errors to census operations to improve future
decennials; in 2000, the coverage measurement program focused on net
coverage error, not components of coverage error. Although the Bureau
will once again estimate net coverage error, Bureau documentation shows
that estimating the components of coverage error is now a higher
priority for the Bureau. Coverage error comprises two types of errors:
errors of omission and errors from a person or housing unit mistakenly
enumerated (erroneous enumerations). The impact of omissions and
erroneous enumerations are combined to determine the net error.
During Census 2000, the Bureau estimated the net coverage error of the
census. However, net error can disguise significant count problems--the
Bureau could have a large overcount in some areas and large undercount
elsewhere, and it might balance out to a small net error nationwide. In
addition, the coverage measurement definitions of erroneous
enumerations used in 2000 were not as precise as those that will be
used in 2010. For example, the 2000 coverage measurement program did
not require Bureau officials to ascertain where a person should have
been counted. Instead, some enumerations were treated as omissions or
erroneous enumerations in an area where they should have been counted.
According to the Department of Commerce, because of the Bureau's new
goal to measure the components of coverage error, the Bureau will use a
definition for component error estimation that is not as strict as that
used for net error. For 2010, erroneous enumerations for the coverage
measurement program are defined to include the following: duplicate
enumerations, persons born after or who die before Census Day, or
persons who are not residents of a housing unit in the United States on
Census Day.
Though the Bureau recognizes the potential benefits of estimating the
components of coverage error to improve future decennials, it has not
fully developed its plans for doing so in 2010. Providing the Congress
and other key stakeholders with information about how the Bureau plans
to use the 2010 coverage measurement program to improve future
decennials would enhance transparency and oversight of coverage
measurement.[Footnote 25] For Census 2000, the Bureau did not associate
the estimates of erroneous enumerations and omissions to specific
census operations. For the 2010 Census, the Bureau has not finalized
plans to evaluate the quality of its estimates of components of
coverage error (i.e., estimates of variance) and has only recently
determined the level of geography for which estimates will be made.
Moreover, although the Bureau has developed table shells that associate
components of coverage error to census operations, the Bureau has not
clearly explained how it plans to implement operations and data
collection strategies that will support the analyses and lead to the
completion of the tables and convert those analyses into plans to
improve the census operation for 2020. Importantly, Bureau officials
stated that it will be difficult to link estimates of the components of
coverage error to specific census operations. Instead, they
characterize the process of associating the components of error to
census operations as an exploratory effort. A Bureau official explained
that associating errors of omission to operations will be most
challenging because omitting persons from the census count could be
caused by errors in multiple operations. For example, a person could
have been omitted because of a bad address or an incorrect enumeration.
It may not be possible to determine which operation contributed to the
omission. In contrast, she said that it may be more feasible to
associate errors of erroneous enumerations to individual census
operations because there is more that they know about the enumerations.
In associating errors to census operations, a Bureau official stated
that the Bureau may use a modeling approach to identify those factors
with the highest probability and greatest significance in predicting
specific erroneous enumeration errors. Additionally, Bureau officials
noted that how the Bureau associates components of coverage error to
census operations cannot be determined without first determining the
final models for estimating the components of coverage error for the
2010 Census. These models cannot be specified without the actual 2010
Census data and therefore will be specified after data collection.
Finally, as recommended by the National Academy of Sciences, Bureau
officials acknowledged that there is ongoing experimental research
involving the use of a master trace sample database in associating the
components of coverage error to census operations. This database is
intended to help the Bureau trace response and operational data through
various stages of the 2010 Census--such as address listing and data
collection. The National Academy of Sciences notes that such a database
should allow the Bureau to retain a full history of relevant census
operations for a designated sample of households in census blocks. The
Bureau should then be able to use the database to investigate the
source of a variety of potential deficiencies that can arise in a
complicated undertaking like the decennial census. More specifically,
the database could be used to determine what percentage of census
omissions are from partially enumerated households and what percentage
of omissions are from the merged administrative records database.
According to a Bureau official, the Bureau plans to begin developing
the database in 2008. However, the official noted several limitations
with such a database. For example, the database consists of a sample of
the U.S. population, as opposed to the whole census population. Still,
Bureau officials noted that improving the master trace sample database
in 2010 is important, citing the example of using the database for
complex cluster analyses that link to coverage measurement.
While most experts we contacted agreed that estimating the components
of coverage error will likely help to improve future decennials, some
experts questioned the Bureau's ability to link the actual errors to
specific census operations. For example, one expert said that it "will
be practically impossible to project errors to operations."
Additionally, some experts discussed other challenges that the Bureau
may face in accurately estimating the components of coverage error,
including distinguishing between different types of erroneous
enumerations and omissions as well as resolving duplicates and
accurately matching the data. Still, some experts did say that it may
be possible for the Bureau to link certain erroneous enumerations to
actual operations. Because of the Bureau's new objective to estimate
components of coverage error and relate them to census operations, it
is important for the Bureau to solidify its plans and share them with
stakeholders to ensure that there is an understanding on what the
program will produce in order to improve future decennials.
Bureau Plans Expanded Matching for Resolving and Identifying Duplicate
Enumerations:
Similar to the matching planned to expand the coverage follow-up
operation, the Bureau's plans for its 2010 coverage measurement program
include expanding the matching process in order to better resolve and
identify duplicate enumerations--a major source of error in the 2000
Census and coverage measurement program. Some of the experts we
contacted are confident that the plans will be an improvement. Shortly
after the census enumeration is completed, a postenumeration survey
will be conducted in a sample of block clusters. This re-enumeration
sample--known as the P-sample--is used with the E-sample (the actual
census enumeration in those sample areas) to derive a corrected
population estimate.[Footnote 26] For 2010, the Bureau will also use
computer matching to compare the P-sample with all census enumerations,
with the objective of identifying and resolving duplicate enumerations.
This is an expansion from 2000, when the P-sample was matched against
enumerations from sample blocks and surrounding blocks only, although
housing units are matched to census housing in the search area. As in
2000, the Bureau will also use clerical matching to do additional
matching, but will also do nationwide matching.[Footnote 27]
Despite the Bureau's effort to improve its matching process and
minimize the number of duplicates in 2010, the Bureau does not yet have
complete plans for the 2008 Dress Rehearsal or for the 2010 Census to
help accomplish this goal. Bureau officials noted that materials are
being developed for certain matching procedures, such as clerical
matching. Further, Bureau officials said that budgetary constraints may
also affect the matching operations and changes may occur before or
following the 2008 Dress Rehearsal. For example, the Bureau may not
conduct person matching during the dress rehearsal.
Though a few experts cited the challenges the Bureau will continue to
face in resolving duplicate enumerations, several also noted that the
expanded matching process should help to reduce the number of
duplicates considerably. An expert explained that by expanding the
geography covered in the Bureau's matching operations, it will be
easier to identify duplicates as well as households that may have been
overlooked in previous coverage measurement matching operations because
the search area was limited to respective census blocks and a few
nearby surrounding geographic areas. Still, he also said that the
expanded matching will be an improvement only if implemented well.
Bureau Plans for Evaluating the 2010 Coverage Follow-up Operation and
Census Coverage Measurement Program Are in Early Stages:
The Bureau is in the early stages of developing plans for evaluating
its 2010 Census coverage follow-up operation and census coverage
measurement program. To assist in this effort, the Bureau has
contracted with the National Academy of Sciences to recommend proposed
research for the 2010 Census. The academy's final report is due in
September 2009.[Footnote 28] In its December 2007 interim report, the
academy recommended that the Bureau conduct certain experiments and
evaluations, such as assessing the use of administrative records and
coverage probes for identifying cases for follow-up.[Footnote 29]
Bureau officials have also suggested research on various aspects of
coverage follow-up and measurement for 2010. Officials said that the
Bureau will consider costs and staffing needs in deciding what it will
evaluate. However, the Bureau has not yet developed a timetable for
making decisions about evaluating the 2010 coverage follow-up operation
or census coverage measurement program.
For the most recent decennial censuses, the Bureau has carried out
experiments and evaluations. A census experiment usually involves field
data collection during the census in which alternatives to current
census processes are used and assessed for a subset of the population.
An evaluation is usually an analysis conducted on data collected as
part of the decennial process to determine whether individual steps in
the census operated as expected. The Bureau plans to work with internal
and external stakeholders on defining and developing its program for
experiments and evaluations for the 2010 Census, with design work and
implementation starting in 2008 and continuing through 2011. The final
set of activities would include analysis, documentation, and
presentation of the research, and these activities would start in 2009
and be completed by the end of the fiscal year 2013.
According to the National Academy of Sciences' December 2007 interim
report, its purpose was to reduce the possible subjects for census
research from a list of over 50 research topics. The academy reviewed
the Bureau's initial list of research topics to help reduce the
possible subjects for census experimentation, and offered broad advice
in its interim report on plans for evaluations of the 2010 Census. For
this smaller number of research topics, the academy recommended several
experiments and evaluations relating to coverage improvement and
measurement. For example, the academy recommended that the Bureau
assess the use of administrative records in the event that the Bureau
opts to review administrative records to identify potential omissions.
It also recommended that the Bureau include one or more alternate
questionnaire experiments, such as an examination of the usefulness of
the residence rules and concepts on the census questionnaire. Further,
the academy recommended a possible experiment on comparing telephone
interviews to personal interviews for the coverage follow-up operation
because a telephone-only operation would follow up for only those
households that provide a contact telephone number. However, at this
point, the Bureau has not yet made definitive decisions about what it
will evaluate and when it will make these decisions.
In its interim report, the National Academy of Sciences suggested that
the Bureau design a master trace sample database--a tool for
investigating the source of potential deficiencies that can arise in
such a complicated undertaking as the decennial census--that would help
assess the benefits of the coverage follow-up interview. This database
could link person, household, and housing unit characteristics; census
processes; and the presence or absence of census component coverage
error. The academy suggested that the use of this database would better
identify the sources of high rates of census component coverage error,
including erroneous enumerations, enumerations in the wrong place,
duplications, and omissions. The academy also suggested that the Bureau
use a reverse record check for coverage measurement. It noted that the
reverse record check is an alternative method for estimating the
completeness of census coverage of the population, which may have
advantages over the methods of dual systems estimation and demographic
analysis that have been used for this purpose to date. This procedure
has not been used previously to evaluate census data, mainly because of
the 10-year period between censuses, which complicates the need to
trace people's addresses from one census to the next. However, the
Bureau now uses the American Community Survey, a monthly survey of
250,000 housing units to collect and provide long-form data on an
annual basis. A reverse check would facilitate comparing decennial
enumerations to information collected more frequently during the
American Community Survey.
The National Academy of Sciences believes that it will be very
important for the Bureau to finalize specific designs of its
experiments by summer 2008 to meet the planning needs of the census.
Because the data needed to support census evaluations are typically
output files from the census itself, the exact structure of individual
evaluations is not yet as time-sensitive as the experiments. However,
the academy notes that some early planning for evaluations is critical
so that the necessary data extracts can be prepared and retained,
especially because much of the data collection in 2010 will be carried
out by contractors, and thus data retention requirements need to be
arranged with contractors as early as possible. It also suggests that
the Bureau increase its in-house expertise in experimental design for
the census. Some experts we interviewed about the Bureau's plans for
coverage measurement also noted that the Bureau should plan its
evaluations with more urgency. An expert noted that in 1990 and 2000,
when the Bureau faced challenges with its adjustment process, there
were fewer staff members available to work on evaluations; he said that
perhaps this would not be a problem for the 2010 Census because the
Bureau did not plan to adjust the census count. Further, a few experts
suggested that the Bureau should ask outside researchers, in addition
to the National Academy of Sciences, for assistance and feedback in its
evaluation planning and development. A few experts also suggested that
the Bureau should consider developing more qualitative evaluations,
such as studies to look deeper into why certain populations, such as
the homeless, are omitted from the census at greater rates.
Conclusions:
With only 2 years remaining before Census Day, the Bureau faces
formidable challenges in successfully implementing and completing the
2010 Census. These challenges are even more formidable given the
Bureau's recent decision to redesign certain aspects of the 2010
Census, and make more critical the need for the Bureau to have in place
specific and concrete plans for ensuring and measuring the accuracy of
the decennial census. To accurately count everyone residing in the
United States, the Bureau must overcome significant challenges due to
the nation's increasing diversity, as well as, according to the Bureau,
a population increasingly reluctant to participate in surveys. In
response to these challenges, the Bureau plans to expand its coverage
follow-up operation--a final quality assurance method intended to help
accurately enumerate the population--by focusing on resolving count
discrepancies, enumerating large households, and following up on
households with potential enumeration problems identified through
techniques the Bureau plans to further test during its dress rehearsal.
As in prior decennials, the Bureau plans to measure the accuracy of the
2010 decennial through a coverage measurement program. The Bureau
intends to focus its coverage measurement efforts in 2010 on learning
more about the components of census error in order to improve
operations during the next decennial--2020--and future decennials. The
Bureau does not plan to use its 2010 coverage measurement program to
adjust the count of the nation's population.
The Bureau has initiated efforts to improve the accuracy of the census
count by expanding coverage follow-up and improving measurement;
however, some areas bear enhanced monitoring, oversight, and management
as the Bureau moves from its 2008 Dress Rehearsal to final plans for
the 2010 Census. Providing internal and external stakeholders a full
description of its plans for coverage follow-up and measurement will
help the Bureau ensure that the plans for and expected outcomes of
these important operations are understood not only by the Bureau but
also by stakeholders. Importantly, many of the operational decisions
for the 2010 Census depend on the upcoming 2008 Dress Rehearsal. For
example, the Bureau has not finalized its plans for 2010 regarding the
scope of its coverage follow-up operation and will base final decisions
on how to select cases for follow-up in part on its experiences in the
dress rehearsal. Coverage follow-up improvements have been tested on a
site-level scale but have not yet been tested with new processing
interfaces between the Bureau and its contractor for collecting and
disseminating information. The Bureau plans to expand coverage follow-
up but has not clearly stated the basis it will use for deciding which
techniques to use or the number of cases it will include for follow-up.
The Bureau will rely on the dress rehearsal to assess the effectiveness
of certain coverage follow-up techniques, such as the undercount probe.
Existing analyses suggest that this probe is of limited use. Overall,
given the uncertainty surrounding key aspects of the Bureau's plans for
coverage follow-up for 2010, as well as the delayed dress rehearsal,
the content and magnitude of the 2010 coverage follow-up bear continued
scrutiny.
Operational plans for coverage measurement lack specificity, and the
2010 Census will be the first time that the Bureau can implement its
expanded computer matching for both coverage follow-up and measurement
on a national level. Importantly, although an announced purpose for
revising the census coverage measurement program was to provide
improvements for the 2020 Census and the Bureau is working toward
associating components of error to specific operations, it has neither
clearly explained how it will do this nor the expected dates for when
its plans for this analysis will be complete. All of these unknown
aspects increase the level(s) of risk of not meeting these important
goals. By having plans and timelines in place, the Bureau increases the
likelihood of completing its coverage measurement analyses in time to
inform the planning for the 2020 decennial.
Because the Bureau plans significant changes to its coverage follow-up
and measurement activities, it should be planning well in advance of
the decennial how it will evaluate and measure their success. Timely
and informed oversight, monitoring, and management can promote the
success of these evaluative activities. The Bureau's program for
evaluations and experiments has great potential to help improve the
next decennial. The National Academy of Sciences has made interim
recommendations on various ways that the Bureau could assess its
decennial data collection activities. Among other things, the academy
recommends using a master trace sample because such a sample would
improve the Bureau's efforts to relate census errors to census
operations. We believe that such a master trace sample has the
potential to help the Bureau better understand sources of error.
Nonetheless, designing such a sample is a daunting task. The Bureau has
begun to evaluate how to implement the academy's recommendation. The
academy's interim report also suggested that the Bureau soon finalize
specific designs of the experiments to meet its planning needs so that
necessary data retention requirements could be arranged with
contractors as early as possible.
Recommendations for Executive Action:
We are recommending that the Secretary of Commerce direct the Bureau to
provide the Congress and other census stakeholders with more specific
plans on conducting coverage follow-up, implementing coverage
measurement, and evaluating the results of those programs for the 2010
Census. Specifically, we recommend that the Bureau provide:
* the criteria it will use to assess the techniques for identifying
cases for the coverage follow-up operation in the 2008 Dress Rehearsal
and the timeline for completing this assessment to permit planning for
2010--the timeline should specify dates for preliminary assessments,
final assessments, and when operational changes might be made to the
2010 Census;
* its plans for conducting coverage measurement for the 2010 Census,
which should include a description of when it will provide estimates of
net coverage error and components of coverage error and how it plans to
relate components of coverage error to census operations in order to
improve future decennials; and:
* key decision points and plans for evaluating aspects of the 2010
Census coverage follow-up operation and census coverage measurement
program.
Agency Comments and Our Evaluation:
We provided a draft of this report to the Secretary of Commerce for his
review and comment. Commerce had no substantive disagreements with our
recommendations and stated that it will develop formal action plans for
each recommendation. While Commerce believes a timeline for producing
estimates of net coverage error and components of coverage error is
certainly a reasonable expectation, Commerce noted that it is unclear
as to how it can at this time provide any specifics on how coverage
results might improve future censuses before the coverage data are
obtained and analyzed. We recognize that the Bureau has developed table
shells that associate components of coverage error to census
operations, but these table shells and plans for how the Bureau will
conduct its analyses have not been shared with the Congress and other
stakeholders. In addition, it is not clear from current Bureau plans
how it will go about obtaining the data required to complete the
tables.
As agreed with your office, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 30 days
after the report date. We will then send copies of this report to the
Secretary of Commerce, the Department of Commerce's Inspector General,
the Director of the U.S. Census Bureau, and interested congressional
committees. We will make copies available to others upon request. This
report will also be available at no charge on our Web site at
[hyperlink, http://www.gao.gov].
If you or your staff have any questions concerning this report, please
contact me at (202) 512-6806 or sciremj@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. GAO staff who made major contributions to
this report are listed in appendix III.
Sincerely yours,
Mathew J. Scirč:
Director, Strategic Issues:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
To determine (1) how the U.S. Census Bureau (Bureau) plans to improve
coverage through its coverage follow-up operation, (2) how the Bureau's
census coverage measurement plans for assessing the accuracy of the
2010 Census compare with efforts in 2000, and (3) how the Bureau plans
to evaluate the coverage follow-up and measurement efforts after the
2010 Census, we requested and obtained source documents from the
Bureau's headquarters in Suitland, Maryland. We also interviewed Bureau
officials in several of its divisions, including the Decennial
Management Division and the Decennial Statistical Studies Division.
Further, we reviewed evaluations regarding coverage follow-up and
coverage measurement.
In addition, for the second objective, related to the Bureau's census
coverage measurement program, we interviewed 15 experts about the
Bureau's coverage measurement methodology and plans. Prior to our
interviews, we summarized the Bureau's plans for coverage measurement
and met with Bureau officials to obtain confirmation of our summary. We
then identified experts based on published reports as well as
recommendations from the Bureau and the National Academy of Sciences
and contacted them about reviewing our summary of the Bureau's
measurement plans and interviewing them about the Bureau's plans,
including the extent to which the Bureau's plans addressed concerns and
limitations of previous coverage evaluation programs, the extent to
which using logistic regression instead of poststratification would
improve the estimation of net coverage error, the main challenges in
estimating components of coverage error, and the implications of
delaying the coverage measurement interviews. Fifteen experts responded
affirmatively to our request for interviews and we subsequently
interviewed the following persons:
Dr. Barbara Bailar:
Statistical Consultant:
Washington, D.C.:
Dr. Thomas Belin:
Professor of Biostatistics, Psychiatry, and Biobehavioral Sciences:
Department of Biostatistics:
School of Public Health:
University of California, Los Angeles:
Los Angeles, California:
Dr. Michael Brick:
Senior Statistician:
Vice President, Director of Survey Methods:
Associate Director of the Statistical Staff at Westat:
Rockville, Maryland:
Dr. James Brown:
Senior Lecturer in Official Statistics:
School of the Social Sciences:
University of Southampton Southampton, United Kingdom:
Mr. Dave Dolson:
Director:
Social Survey Methods Division:
Statistics Canada:
Ottawa, Canada:
Dr. Stephen Fienberg:
Maurice Falk:
University Professor of Statistics and Social Science:
Department of Statistics:
Carnegie Mellon University:
Pittsburgh, Pennsylvania:
Dr. Tim Johnson:
Director:
Survey Research Laboratory:
University of Illinois at Chicago:
Chicago, Illinois:
Mr. Don Royce:
Director General:
Methodology Branch:
Statistics Canada:
Ottawa, Canada:
Dr. Duane Steffey:
Director:
Statistical and Data Sciences:
Exponet, Inc.:
Menlo Park, California:
Dr. John Thompson:
Executive Vice President of Survey Operations:
National Opinion Research Center at the University of Chicago:
Chicago, Illinois:
Dr. Robert Tortora:
Chief Methodologist:
Gallup Washington:
Washington, D.C.:
Dr. Roger Tourangeau:
Director:
Joint Program in Survey Methodology:
University of Maryland:
College Park, Maryland:
and:
Acting Director:
Program in Survey Methodology:
University of Michigan:
Ann Arbor, Michigan:
Dr. Kenneth Wachter:
Professor of Demography and Statistics:
University of California at Berkeley:
Berkeley, California:
Dr. Martin Wells:
Director of Research:
Charles A. Alexander:
Professor of Statistical Sciences:
Cornell University:
Ithaca, New York:
Dr. Kirk Wolter:
Head of the Center for Excellence in Survey Research:
Professor in the Department of Statistics:
Senior Fellow at the National Opinion Research Center:
University of Chicago Chicago, Illinois:
We conducted this performance audit from January 2007 to March 2008 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
[End of section]
Appendix II: Comments from the Department of Commerce:
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
The Secretary Of Commerce:
Washington, D.C. 20230:
April 3, 2008:
Mr. Mathew J. Scire:
Director:
Strategic Issues:
United States Government Accountability Office:
Washington, DC 20548
Dear Mr. Scire:
The U.S. Department of Commerce appreciates the opportunity to comment
on the United States Government Accountability Office's Draft Report
Entitled Census 2010: Bureau Needs to Specify How It Will Assess
Coverage Follow-up Techniques and When It Will Produce Coverage
Measurement Results, (GAO-08-414). I enclose the Department's comments
on this report.
Sincerely,
Signed by:
Carlos M. Gutierrez:
Enclosure:
U.S. Department of Commerce Comments on the United States Government
Accountability Office Draft Report Entitled Census 2010: Bureau Needs
to Specify How It Will Assess Coverage Follow-up Techniques and When It
Will Produce Coverage Measurement Results, (GAO-08-414) March 2008:
The U.S. Department of Commerce appreciates the United States
Government Accountability Office's (GAO) efforts to review our coverage
improvement and coverage measurement processes planned for the 2010
Census and this opportunity to review the draft report.
We have no substantive disagreements with these recommendations. Once
the report is issued in final form, we will develop formal action plans
for each recommendation. While we believe a timeline for producing the
estimates is certainly a reasonable expectation (Recommendation 2), we
are unclear as to how we could at this time provide any specifics on
how our coverage results might improve future censuses before the
coverage data are obtained and analyzed by Census Bureau and other
experts.
Specific Comments on the Draft Report:
Page - Highlights: Item 4 in the second paragraph reads (4) expand
computer matching to match housing units and persons in its post
enumeration survey to the entire census.
(See comment 1.):
Comment: The item should be revised to read (4) expand computer and
clerical matching to match persons in its post enumeration survey to
the entire census. The nationwide matching is not done for housing
units.
Page 4: The error above regarding expanding computer matching also
appears on page 4 of your report and should be revised.
(See comment 2.):
Page 4: For clarification, there is important information that needs to
be included about the coverage measurement interviewing starting four
months later than in 2000. The telephone phase was dropped due to the
increased complexity of the person interview instrument, and loss of
time in the conversion from a telephone phase to personal-visit
implementation. Part of the delay in starting the 2010 Person Interview
operation is due to eliminating the telephone phase. (For Census 2000,
the person field visit started on 6/1/00, while the telephone phase
started on 4/24/00). Although the interviewing will start four months
later compared to the Census 2000 phone phase, person interviewing ends
only one and a half months later than in 2000 (9/1/2000 vs.
10/16/2010). This should reduce the concerns related to recall bias.
(See comment 3.):
Page 4, footnote 2: The footnote reads, The dual systems approach .
estimates of the remaining error in the count after omissions and
erroneous enumerations offset one another. Comment: The second sentence
may give a wrong impression that a net error estimate requires
estimates of omissions and erroneous enumerations. To be more accurate,
we usually say "reflects" rather than "estimates." We would suggest
correcting the sentence to read, The net error reflects the error
remaining after omissions and erroneous enumeration offset one another.
(See comment 4.):
Page 4, footnote 3: The footnote reads, "Before computer matching is
completed, the Bureau conducts . to estimate errors in the census. such
as nonexistent persons or duplicates."
(See comment 5. Now footnote 4, p4..):
Comment: We offer several clarifications to this footnote. A post
enumeration survey consists of two samples: the P and E samples. P-
sample persons are matched not only to the E sample, but also to the
entire census. E-sample persons are matched to the entire census to
identify and resolve duplicates. E-sample housing units are matched to
census enumerations in the block cluster and surrounding ring of
blocks.
Page 16: For clarification, as written, the third paragraph implies
that when the nonresponse follow-up (NRFU) undercount probe tells us
there may be an additional person who was not included on the NRFU
roster, the coverage follow-up (CFU) interview will clarify the
residency situation so that a person who should be included (but was
not on the NRFU roster) will be added in CFU. The report describes this
as "included in the CFU roster after NRFU," which is not accurate. We
do not add these persons and collect these data during NRFU. If CFU
roster clarifications indicate someone should be added, this happens
during CFU.
(See comment 6. Now footnotes 11, p. 16..):
Page 18: For clarification of the first paragraph, the limitation with
identifying duplicates in site tests is not so much the limited sample
size of the test sites, but rather the limited likelihood of
duplication of individuals that will occur within a site of limited
geographic size.
(See comment 7.):
Page 18: We suggest two clarifications to the second paragraph: (1)
duplicated housing units occur when multiple addresses or
representations of an address for the same housing unit appear on the
address list. The ones that are "identified during census operations"
are removed from the list for subsequent operations and (2) the next
sentence about units being included on both the housing unit list and
group quarters list is no longer accurate, which represents a big
change from Census 2000. We now have one list for both, with units
identified as either housing units or group quarters. Therefore, in
general, we can no longer have the situation described in the document
that "addresses can be included in the census in both the housing unit
and group quarters enumerations."
(See comment 8. Now on p. 19..):
Page 19: We suggest two clarifications to the second paragraph: (I) the
description of the unduplication process is a bit reversed. During the
automated matching process, a match score is assigned to all links
based on the level of match. The Census Bureau assesses the quality of
the match based on the match score and (2) the Field Verification
operation verifies housing unit status of the units selected for follow-
up in the operation. Units that are invalid are marked as deletes or
duplicates, and valid units are marked as verified. There are no
additions during the Field Verification. In some cases, the Field
Verification operation verifies the existence of units added in other
operations, in particular, the Be Counted program. For the housing unit
unduplication cases, the units are already on the list, and Field
Verification is undertaken in order to determine if housing units
should be deleted from the list.
Page 20: For clarification of the second paragraph, the Census Bureau
plans to use all our CFU test results to make final decisions about
what to implement in 2010. not just the 2008 results. In particular, we
have results from the 2005 National Content Test and 2006 Census Test.
The 2008 implementation was intended to be an effort to collect
additional data.
(See comment 10.):
Page 22: Your statement reads, "Finally, to better measure coverage,
the Bureau plans to better identify duplicates by expanding its
computer matching."
(See comment 11.):
Comment: While true, we also will expand clerical matching. The
following revision is suggested: "Finally, to better measure coverage,
the Bureau plans to better identify duplicates by expanding its
computer and clerical matching."
Page 23: The first sentence reads, "During this operation, Bureau field
staff interview” in person”residents of these housing units and collect
demographic and Census Day residency information of all persons living
at the housing unit."
Comment: The sentence should be revised to read as follows: "During
this operation, Census Bureau field staff interview”in person”residents
of these housing units and collect demographics, Interview Day, and
Census Day residency information of all persons living at the housing
unit. Additional information will be collected in 2010 to identify
alternate addresses where the coverage sample roster people may have
been counted on Census Day. This information will be used in person-
based computer and clerical matching operations."
(See comment 12.):
Page 23: The fourth sentence in the first paragraph reads,
"Additionally, the Bureau conducts computer matching to further resolve
cases."
(See comment 13.):
Comment: This statement is inaccurate. A computer match is not done
after we conduct the follow-up.
Page 23: The last sentence in the first paragraph reads, "However,
unlike for Census 2000, the Bureau will not produce state-level
estimates by race/ethnicity, age/sex, and tenure or estimates lower
than the state level."
(See comment 14.):
Comment: This statement is incorrect. The Census Bureau did not produce
any state- level estimates broken down by any characteristics in Census
2000.
Page 26: The third bullet also reflects the same error as occurred on
the Highlights page and page 4. Refer to the comment made on the
Highlights page above.
(See comment 15.):
Page 28, first full paragraph: For clarification, the Census Bureau has
not "redefined what constitutes an erroneous enumeration" because of
any shortcomings. Instead, because of the new goal to measure the
components of coverage error, it became important to use (for component
error estimation) a definition that is not as strict as that used for
net error. But the strict definition will continue to be used for net
error in 2010.
(See comment 16. Now p. 29..):
Page 31, second paragraph: For clarification, technically speaking, the
P-sample is not used to estimate the correct population, but is used to
estimate the match rate, while the E-sample is used to estimate the
correct enumeration rate. Both of these samples are needed to estimate
the true population. Matching is performed to the census, not the E-
sample, although housing units are only matched to census housing in
the search area.
(See comment 17. Now footnote 26, p. 31):
Page 31, footnote 25: Your footnote reads, "Bureau officials have noted
that the clerical matching procedures being planned for 2010 are
comparable to those used for coverage measurement in the 2000 Census."
(See comment 18. Now footnote 27, p. 32..):
Comment: The clerical matching procedures for 2010 are much more
complex than in 2000. We have to account for the duplicate searching at
alternate respondent-provided addresses collected in the person
interviews and person follow-up interviews. So they are not strictly
comparable.
Pages 31-32, last sentence: Your statement reads, "Although the Bureau
will not include the coverage measurement person follow-up interviews
in the dress rehearsal, for 2010 the Bureau plans to use information
collected from those interviews as well as initial person interviews in
its computer-matching operation."
(See comment 19.):
Comment: This statement is inaccurate. Information collected from
Person Follow-up is not used in computer matching, but only in clerical
matching.
Page 33: The last paragraph references 52 research topics covering all
aspects of its decennial operations that the Bureau provided for the
National Academy of Sciences to review.
Comment: The paragraph is misleading and should be clarified as
follows: The Census Bureau provided a list of 54 research questions
sorted among 11 research topics to the National Academy of Sciences for
their review. For the most part, the research questions were not
identified as either evaluations or experiments. The Academy did not
reduce the list of subjects for census experimentation to about six.
Rather, the Academy's interim report presents its ideas for 2010 Census
experiments, which in some cases are similar to the Census Bureau's
proposed research.
(See comment 20.):
The following are our comments on the Department of Commerce's letter
dated April 3, 2008.
GAO Comments:
We clarified our recommendation on relating components of coverage
error to census operations in order to improve future decennials.
Specifically, we recognize that the Bureau has developed table shells
that associate components of coverage error to census operations, but
these table shells and plans for how the Bureau will conduct its
analyses should be shared with the Congress and other stakeholders. In
addition, it is not clear from current Bureau plans how it will go
about obtaining the data required to complete the tables.
1. We have revised the Highlights page to reflect the Department of
Commerce's clarification that for the census coverage measurement
program, the Bureau will expand its computer and clerical matching to
match persons in its postenumeration survey.
2. We have revised the report to reflect the Department of Commerce's
clarification that the Bureau will expand its computer and clerical
matching to match persons in its postenumeration survey.
3. In footnote 3, we have revised the report to clarify that the
although the coverage measurement interview will start 4 months later
compared to the Census 2000 telephone phase, person interviewing for
the 2010 Census ends 1-1/2 months later than in 2000.
4. In footnote 2, we have revised the sentence to indicate that the net
error reflects the error remaining after omissions and erroneous
enumerations offset one another.
5. In footnote 4, we have revised the definition of the P-sample and E-
sample based on the Department of Commerce's clarification.
6. On page 16, we have clarified information about using the undercount
probe during nonresponse follow-up and coverage follow-up.
7. On page 18, we have clarified the information about the limited
likelihood of duplication of individuals that will occur within a site
of limited geographic size.
8. On page 19, we have incorporated the clarifying information provided
by the Department of Commerce.
9. On page 19, we have clarified information about the matching
process.
10. On page 20, we have revised the report to reflect that the 2008
Dress Rehearsal is intended to provide additional data to make final
decisions about the Bureau's implementation of coverage follow-up in
2010.
11. On page 22, we have revised the report to reflect that the Bureau
will expand its computer and clerical matching to match persons in its
postenumeration survey.
12. On page 23, we have incorporated the Department of Commerce's
revisions on information that is collected during the census coverage
measurement program.
13. On page 23, we have deleted the incorrect sentence.
14. On page 23, we have deleted the incorrect sentence.
15. On page 26, we have revised the sentence to reflect that the Bureau
will expand its computer and clerical matching to match persons in its
postenumeration survey.
16. On page 29, we revised the sentence to clarify the reason for
refining the Bureau's definition for erroneous enumeration.
17. On page 31, we clarified the definition of the P-sample and E-
sample as suggested.
18. On page 31, for footnote 26, we acknowledged the change in clerical
matching for the 2010 Census.
19. On page 31, we deleted the incorrect sentence.
20. On page 33, we provided attribution for the information to the
National Academy of Sciences' December 2007 interim report.
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Mathew J. Scirč (202) 512-6806 or sciremj@gao.gov:
Acknowledgments:
In addition to the contact name above, Ron Fecso, Chief Statistician;
Ernie Hazera, Assistant Director; Jenna Aurand; Betty Clark; and
Jennifer Edwards made key contributions to this report. Tom Beall,
Andrea Levine, Donna Miller, Lisa Mirel, and Elizabeth Wood provided
significant technical support.
[End of section]
Footnotes:
[1] The Bureau is planning to use a 300,000 national housing unit
sample.
[2] The dual systems approach uses a postenumeration survey to produce
an estimate of the true population, which is then used to produce an
estimate of the net coverage error. The net error reflects the error
remaining after omissions and erroneous enumerations offset one
another. See GAO, 1990 Census Adjustment: Estimating Census Accuracy--
A Complex Task, GAO/GGD-91-42 (Washington, D.C.: Mar. 11, 1991).
[3] Although the interviewing will start 4 months later compared to the
Census 2000 telephone phase, person interviewing for the 2010 Census
ends 1-1/2 months later than in 2000.
[4] The Bureau conducts a postenumeration survey, independent of the
census, in a sample of block clusters. This sample is known as the P-
sample and estimates the number of persons missed by the census. A
postenumeration survey consists of two samples: the P-sample and the E-
sample. P-sample persons are matched not only to the E-sample, but also
to the entire census. E-sample persons are matched to the entire census
to identify and resolve duplicates. E-sample housing units are matched
to census enumerations in the block cluster and surrounding ring of
blocks.
[5] The coverage measurement interview depends upon the respondents'
ability to recall the composition of their households on Census Day.
The longer the period of time between Census Day and the day that the
measurement interview is conducted, the harder it is for respondents to
recall membership of their households on Census Day.
[6] For example, see GAO, 2010 Census: Cost and Design Issues Need to
Be Addressed Soon, GAO-04-37 (Washington, D.C.: Jan. 15, 2004); 2010
Census: Basic Design Has Potential, but Remaining Challenges Need
Prompt Resolution, GAO-05-9 (Washington, D.C.: Jan. 12, 2005); and 2010
Census: Census Bureau Should Refine Recruiting and Hiring Efforts and
Enhance Training of Temporary Field Staff, GAO-07-361 (Washington,
D.C.: Apr. 27, 2007).
[7] Starting in January 2007, the Bureau notified state and local
governments that it would seek their help in developing a complete
address file through the Local Update of Census Addresses program.
Address canvassing--a field operation to build a complete and accurate
address list in which census field workers go door to door verifying
and correcting addresses for all households and street features
contained on decennial maps--will begin in April 2009. For more
information, see GAO, 2010 Census: Census Bureau Has Improved the Local
Update of Census Addresses Program, but Challenges Remain, GAO-07-736
(Washington, D.C.: June 14, 2007).
[8] See also National Academy of Sciences, Once, and Only Once, and in
the Right Place: Residence Rules in the Decennial Census (Washington,
D.C.: 2006).
[9] A household that completes a coverage follow-up interview could
either finish the interview with the same roster of household members
listed at the beginning of the interview or make some change to the
roster. A roster change occurs when a household adds at least one
person to the original roster, deletes at least one person from the
original roster, or both.
[10] Of the 4,191 completed interviews, at least one person was added
to 360 households while at least one person was deleted from 531
households.
[11] During the nonresponse follow-up operation, if persons reply
affirmatively to the undercount probe, the census field workers collect
their names and demographic information. Then, these persons appear on
the coverage follow-up roster and the coverage follow-up interview
confirms the Census Day residency for these persons.
[12] While only 8 persons were included in the coverage follow-up
roster after nonresponse follow-up, the coverage follow-up added an
additional 69 persons from households that were initially enumerated
through nonresponse follow-up.
[13] Bureau officials said that its administrative records system uses
a unique identifier in place of the Social Security number. Social
Security numbers are only used for the purpose of validating the
Bureau's administrative records system.
[14] If the records do not match, the Bureau does not inform the other
agencies because of issues related to census confidentiality.
[15] In the 2004 Census Test, the Bureau was successful in identifying
duplicates using person matching, but it discovered operational
complications with trying to resolve person and housing situations
within the same operation. Basically, the Bureau concluded that with
such a complicated interview, it was too much to ask of the
interviewers that they understand the distinction between person and
housing duplication. A major component of the 2006 Census Test was to
find a way to determine before follow-up whether person duplication was
resulting from housing-level issues or person-level issues.
[16] See GAO, Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1 (Washington, D.C.: Nov. 1999).
[17] See GAO, Internal Control Management and Evaluation Tool, GAO-01-
1008G (Washington, D.C.: Aug. 2001).
[18] Data contamination occurs when a person's inclusion or exclusion
from one census operation affects the probability of his or her
inclusion in a second independent census operation.
[19] The experts were not aware of these results because the documents
were for internal Bureau use only.
[20] According to Bureau officials, the dress rehearsal timeline is the
best current indicator of the timeline for the 2010 Census. However,
Bureau officials have told us that coverage measurement estimation will
not be done for the 2008 Dress Rehearsal. In addition, the Bureau will
not include its coverage measurement housing unit field operation,
housing matching operation, or person follow-up. The Bureau is also
considering dropping person matching from the dress rehearsal.
[21] The exact date is yet to be determined.
[22] Poststratification entailed calculating dual systems estimates
separately for different demographic and geographic groups. Dual
systems estimation requires that a postenumeration survey be conducted.
It produces an estimate of the true population, which is then used to
produce an estimate of the net coverage error. In the 2000 coverage
measurement program, over 400 strata or groups were examined.
[23] For instance, logistic regression allows for the use of continuous
variables like age, instead of only the categorical variables used in
poststratification. Additionally, unlike poststratification, logistic
regression is able to include or omit interactions between variables.
[24] See GAO-01-1008G.
[25] See GAO-01-1008G.
[26] According to the Department of Commerce, the P-sample is used to
estimate the match rate, while the E-sample is used to estimate the
correct enumeration rate. Both of these samples are needed to derive a
corrected population estimate. Matching is performed to the census, not
the E-sample.
[27] Bureau officials have noted that clerical matching procedures
being planned for 2010 are largely comparable to those used for
coverage measurement in the 2000 Census although, with additional
information being collected in 2010, the Bureau will have to account
for duplicate searching at alternate respondent-provided addresses
collected in the person interviews and person follow-up interviews.
[28] The National Academy of Sciences has convened a panel of experts
to review the Bureau's program of research, evaluation, and
experimentation for the 2008 Dress Rehearsal and the 2010 Census. The
panel will consider priorities for evaluation in the 2010 Census. The
panel will conduct its work over a 3-year period, from September 2006
to September 2009.
[29] National Academy of Sciences, Experimentation and Evaluation Plans
for the 2010 Census: Interim Report (Washington, D.C.: Dec. 7, 2007).
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
