2010 Census
Counting Americans Overseas as Part of the Decennial Census Would Not Be Cost-Effective Gao ID: GAO-04-898 August 19, 2004The U.S. Census Bureau (Bureau) has typically counted overseas members of the military, federal civilian employees, and their dependents. However, it usually excluded private citizens residing abroad. In July 2004, the Bureau completed a test of the practicality of counting all overseas Americans. GAO was asked to assess (1) whether the Bureau implemented the test consistent with its design, and (2) the lessons learned from the test results.
The Bureau generally implemented the overseas census test on schedule and consistent with its research design. Still, participation was poor, with just 5,390 questionnaires returned from the three test sites--France, Kuwait, and Mexico. Moreover, because of the low response levels, obtaining those questionnaires proved to be quite expensive--around $1,450 per response, which is far costlier on a unit basis than the 2000 Census. Although the two are not directly comparable because the 2000 Census included operations not used in the overseas test, the 2000 Census cost around $56 per household. Further, boosting the response rate globally might not be practical. On the domestic front, during the 2000 Census, the Bureau spent $374 million on a months-long publicity campaign that consisted of television and other advertising that helped yield a 72-percent return rate. Replicating this level of effort on a worldwide basis would be difficult, and still would not produce a complete count. Ensuring a smooth overseas count could also stretch the Bureau's resources. For example, at each test site the Bureau encountered various challenges that needed to be resolved such as French privacy laws. Moreover, managing a complex operation from thousands of miles away also proved difficult. The approach used to count the overseas population in the 2004 test--a voluntary survey that largely relies on marketing to secure a complete count, lacks the basic building blocks of a successful census. The Bureau has done some initial research on alternatives, but all require more extensive review. Given that the Bureau already faces the difficult task of securing a successful stateside count in 2010, having to simultaneously count Americans abroad would only add to the challenges facing the Bureau.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-898, 2010 Census: Counting Americans Overseas as Part of the Decennial Census Would Not Be Cost-Effective
This is the accessible text file for GAO report number GAO-04-898
entitled '2010 Census: Counting Americans Overseas as Part of the
Decennial Census Would Not Be Cost-Effective' which was released on
September 14, 2004.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Subcommittee on Technology, Information Policy,
Intergovernmental Relations and the Census, Committee on Government
Reform, House of Representatives:
August 2004:
2010 CENSUS:
Counting Americans Overseas as Part of the Decennial Census Would Not
Be Cost-Effective:
GAO-04-898:
GAO Highlights:
Highlights of GAO-04-898, a report to the Subcommittee on Technology,
Information Policy, Intergovernmental Relations and the Census,
Committee on Government Reform, House of Representatives:
Why GAO Did This Study:
The U.S. Census Bureau (Bureau) has typically counted overseas members
of the military, federal civilian employees, and their dependents.
However, it usually excluded private citizens residing abroad. In July
2004, the Bureau completed a test of the practicality of counting all
overseas Americans. GAO was asked to assess (1) whether the Bureau
implemented the test consistent with its design, and (2) the lessons
learned from the test results.
What GAO Found:
The Bureau generally implemented the overseas census test on schedule
and consistent with its research design. Still, participation was poor,
with just 5,390 questionnaires returned from the three test sites”
France, Kuwait, and Mexico. Moreover, because of the low response
levels, obtaining those questionnaires proved to be quite expensive”
around $1,450 per response, which is far costlier on a unit basis than
the 2000 Census. Although the two are not directly comparable because
the 2000 Census included operations not used in the overseas test, the
2000 Census cost around $56 per household. Further, boosting the
response rate globally might not be practical. On the domestic front,
during the 2000 Census, the Bureau spent $374 million on a months-long
publicity campaign that consisted of television and other advertising
that helped yield a 72-percent return rate. Replicating this level of
effort on a worldwide basis would be difficult, and still would not
produce a complete count. Ensuring a smooth overseas count could also
stretch the Bureau‘s resources. For example, at each test site the
Bureau encountered various challenges that needed to be resolved such
as French privacy laws. Moreover, managing a complex operation from
thousands of miles away also proved difficult.
Enumerating Americans in Mexico and France:
[See PDF for image]
[End of figure]
The approach used to count the overseas population in the 2004 test”a
voluntary survey that largely relies on marketing to secure a complete
count, lacks the basic building blocks of a successful census. The
Bureau has done some initial research on alternatives, but all require
more extensive review. Given that the Bureau already faces the
difficult task of securing a successful stateside count in 2010, having
to simultaneously count Americans abroad would only add to the
challenges facing the Bureau.
What GAO Recommends:
Congress may wish to consider eliminating funding for additional
research related to counting Americans abroad as part of the decennial
census, including funding for tests planned in 2006 and 2008. However,
funding for the evaluation of the 2004 test should continue as planned,
particularly to inform congressional decision making on this issue.
Should Congress desire better data on overseas Americans for certain
policymaking and other nonapportionment purposes, Congress may wish to
consider funding research on the feasibility of counting this group
using alternatives to the decennial census. To facilitate this, we
recommend that the Bureau, in consultation with Congress, research
options such as a separate survey, administrative records, and data
exchanges with other countries‘ statistical agencies. The Bureau
agreed with our conclusions and recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-04-898.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Patricia A. Dalton at
(202) 512-6806 or daltonp@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Scope and Methodology:
2004 Overseas Test Was Generally Implemented as Designed:
Overseas Census Test Results Were Disappointing and Costly:
Conclusions:
Matters for Congressional Consideration:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix:
Appendix I: Comments from the Department of Commerce:
Tables:
Table 1: Key Components of Overseas Enumeration Were Generally
Implemented as Planned:
Table 2: Comparison of Responses Received for 2004 Overseas Census
Test:
Figures:
Figure 1: Key Decisions Facing Congress on Enumerating Americans
Abroad:
Figure 2: Census Bureau Ads Placed in the International Herald Tribune:
Figure 3: Census Materials Were Prominently Displayed in Various
Locations in France and Mexico:
Figure 4: Key Building Blocks of a Successful Census:
Letter August 19, 2004:
The Honorable Adam H. Putnam:
Chairman:
The Honorable Wm. Lacy Clay:
Ranking Minority Member:
Subcommittee on Technology, Information Policy, Intergovernmental
Relations and the Census:
Committee on Government Reform:
House of Representatives:
Although more than four million American citizens are believed to
reside abroad, the precise number of overseas Americans is unknown. The
U.S. Census Bureau (Bureau), the federal agency tasked with counting
the nation's population every 10 years, has generally included in the
census overseas members of the military, federal civilian employees,
and their dependents (a group known collectively as "federally
affiliated" individuals), but has typically excluded private citizens
such as retirees, students, and business people.[Footnote 1]
Under the Constitution and federal statutes, the Bureau has discretion
over whether to count Americans abroad. However, in recent years, the
Bureau's policy of excluding private citizens from the decennial census
has been called into question. For example, advocates of an overseas
census claim that better demographic data on this population group
would be useful for a variety of policy-making and business purposes,
and would better represent their unique interests in Congress.
Moreover, in January 2001, Utah sued the Bureau's parent agency, the
U.S. Department of Commerce, claiming that it lost a congressional seat
because the 2000 Census excluded the state's 11,000 Mormon missionaries
and other private citizens living abroad.[Footnote 2] According to the
Congressional Research Service, Utah would have gained a congressional
seat had an additional 855 people been included in the state's
apportionment totals.[Footnote 3] Utah's suit, however, was
unsuccessful.
As we noted in our May 2004 report on this issue, counting Americans
abroad as an integral part of the 2010 Census would be a monumental
task that would introduce new resource demands, risks, and
uncertainties to an endeavor that was already facing a variety of
difficulties.[Footnote 4] Specific challenges include policy questions
such as who should be counted and how should the data be used, as well
as logistical difficulties such as ensuring a complete count and
verifying U.S. citizenship.
To assess the practicality of counting overseas Americans, the Bureau
held a test enumeration from February through July 2, 2004, in France,
Kuwait, and Mexico. As agreed with your offices, we assessed (1)
whether the Bureau implemented the test consistent with its design, and
(2) the initial lessons learned from the test results and their
implications for future overseas enumerations.
To address these objectives, we reviewed applicable planning and other
documents, and interviewed knowledgeable Bureau officials and
representatives of private organizations who helped the Bureau promote
the census at the three test sites. Further, to review how the Bureau
was implementing the census at the test locations, we made on-site
inspections in Paris, France, and Guadalajara, Mexico. We conducted our
work from March through July 2004, in accordance with generally
accepted government auditing standards.
Results in Brief:
The Bureau generally implemented the 2004 overseas census test
consistent with its research design. Key elements of the Bureau's plan,
such as developing a questionnaire specifically for overseas Americans
and launching a marketing campaign designed to publicize the test, were
generally carried out as planned. Moreover, the test was conducted on
schedule.
However, the response levels fell far short of what the Bureau planned
for relative to the number of questionnaires it printed. For example,
although the Bureau printed about 520,000 census forms for the three
test sites--France, Kuwait, and Mexico--the actual number of paper
responses it received only totaled 1,783, as of the end of the test in
early July 2004. Another 3,607 responses were received via the
Internet. Further, because of the low response levels, the data were
expensive to obtain on a unit cost basis--around $1,450 per return. In
contrast, the unit cost of the 2000 Census was about $56 per household.
Although the 2000 Census costs are not directly comparable to the 2004
overseas test because the 2000 test included operations not used in the
overseas test, the 2000 Census was the most expensive census in our
nation's history.
Further, substantially boosting overseas response levels might be
infeasible. For example, during the 2000 Census, the Bureau spent $374
million on a months-long publicity campaign that consisted of
television, radio, and other forms of advertising that helped secure a
72-percent return rate. Replicating this level of effort on a worldwide
basis would be impractical at best, and would not produce a complete
count. Indeed, even after the Bureau's aggressive publicity effort in
2000, it still needed to follow-up with about 42 million households
that did not complete their census forms.
Ensuring a smooth overseas count could also stretch the Bureau's
resources, and thus detract from domestic efforts. For example, at each
test site the Bureau encountered various difficulties that needed to be
worked out. The difficulties included addressing French privacy laws
and delivery problems in Kuwait. Moreover, managing a complex operation
from thousands of miles away was also hard. This was particularly
evident in the logistical challenges the Bureau had in overseeing the
performance of the private firm hired to publicize the census at the
three test sites.
The Bureau's longstanding experience in counting the nation's
population has made it clear that a cost-effective census is assembled
from key building blocks that include mandatory participation, a
complete and accurate address list, and the ability to follow-up with
nonrespondents. The approach the Bureau used to count the overseas
population--a voluntary survey that relies heavily on marketing to
secure a complete count--largely for reasons of practicality, lacks
these building blocks, and it is unlikely that any refinements to this
basic design would produce substantially better results. What's more,
the Bureau already faces the difficult task of securing a successful
stateside count in 2010. Having to count Americans abroad would only
add to the challenges facing the Bureau.
Given the obstacles to a cost-effective count of overseas Americans as
part of the decennial census, Congress may wish to consider eliminating
funding for the research, planning, and development activities related
to counting this population group using the Bureau's current approach,
including funding for tests planned in 2006 and 2008. However, funding
for the evaluation of the 2004 test should continue as planned because
it could provide useful data to Congress. Moreover, should Congress
determine that a count of overseas Americans might be useful for
certain policy-making and other nonapportionment purposes, Congress may
wish to consider authorizing and funding research on the feasibility of
counting Americans abroad using alternatives to the decennial census.
To facilitate congressional decision making, we recommend that the
Bureau, in consultation with Congress, research potential alternatives
to the decennial census such as conducting a separate survey, examining
how administrative records could be refined to produce a more accurate
count of overseas Americans, and exchanging data with other countries'
statistical agencies and censuses, subject to applicable
confidentiality and other provisions. Once Congress knows the tradeoffs
of these various options, it would be better positioned to provide the
Bureau with the direction it needs so that the Bureau could then
develop and test an approach that meets congressional requirements at
reasonable resource levels.
The Secretary of Commerce forwarded written comments from the U.S.
Census Bureau on a draft of this report. The comments are reprinted in
the appendix.
The Bureau agreed with our conclusions and recommendations.
Furthermore, the Bureau noted, "should Congress request and fund"
further research on counting overseas Americans, it would be equipped
to do that research itself.
Background:
According to the Bureau, no accurate estimate exists of the total
number of Americans living abroad. The Constitution and federal law
give the Bureau discretion to decide whether to count American citizens
living abroad. In prior censuses, the Bureau has generally included
"federally affiliated" groups--members of the military and federal
employees and their dependents--but has excluded private citizens
residing abroad from all but the 1960 and 1970 Censuses. The 2000
Census, using administrative records, found 576,367 federally
affiliated Americans residing overseas, including 226,363 military
personnel, 30,576 civilian employees, and 319,428 dependents of both
groups.
In response to congressional direction and the concerns of various
private organizations, the Census Bureau launched a research and
evaluation program to assess the practicality of counting both private
and federally affiliated U.S. citizens residing abroad. The key part of
this effort, the enumeration, took place from February 2004 through
July 2, 2004, in France, Kuwait, and Mexico. To promote the overseas
census test the Bureau relied on third parties--American organizations
and businesses in the three countries--to communicate to their members
and/or customers that an overseas enumeration of Americans was taking
place and to make available to U.S. citizens either the paper
questionnaire or Web site address.
Currently, the Bureau is processing and analyzing the overseas
questionnaire data and plans to complete an evaluation of the test
results in early 2005. The Bureau estimates that it will have spent
approximately $7.8 million in fiscal years 2003 through 2005 to plan,
conduct, and evaluate the 2004 test. The Bureau has requested
additional funds for fiscal year 2005 to plan for further testing
scheduled for 2006. The Bureau also plans to include overseas testing
in the 2008 dress rehearsal if it were to receive the necessary
funding.[Footnote 5]
In May 2004 we reported on the design of the overseas enumeration test
and concluded that because of various methodological limitations, the
test results will only partially answer the Bureau's key research
objectives concerning feasibility (as measured by such indicators as
participation and number of valid returns), data quality, and cost.
Further, we noted that the Bureau should not decide on its own whether
or not to enumerate Americans overseas, and in fact would need
congressional guidance on how to proceed. As shown in figure 1, the key
decisions facing Congress in this regard include, in addition to the
threshold question of whether American residing overseas should be
counted, how the data should be used and whether to enumerate this
population group as part of the decennial census.
Figure 1: Key Decisions Facing Congress on Enumerating Americans
Abroad:
[See PDF for image]
[End of figure]
We also recommended that if further testing were to occur, that the
Bureau resolve the shortcomings of the design of the 2004 test and
better address the objectives of an overseas enumeration.
Scope and Methodology:
As agreed with your offices, our objectives for this report were to
assess (1) whether the Bureau implemented the test consistent with its
design, and (2) the initial lessons learned from the test results and
their implications for future overseas enumerations. To assess the
first objective, we interviewed Bureau officials and compared the
Bureau's test plans with what was actually done at the three test
sites. We visited Paris, France, and Guadalajara, Mexico, to obtain the
views of 12 private, civic, and other organizations on the
implementation of the overseas census test, and/or to confirm at 36
organizations the availability of census material. In addition, to a
more limited extent, we interviewed officials from third party
organizations in Kuwait via the telephone or e-mail. We judgmentally
selected these organizations because they had agreed to display census
promotional materials and, in some cases, had also agreed to do one or
more of the following activities: make available paper copies of the
census questionnaire, publish information in a newsletter, post a link
to a Web site, send outreach e-mail to members, and/or create speaking
opportunities to discuss the census. The results of these visits are
not necessarily representative of the larger universe of third-party
organizations.
To assess the implications of the test results on future overseas
enumerations and the 2010 census, we obtained from Bureau officials
preliminary results of the overseas census by test site and response
mode as well as cost data. We also interviewed officials from the
Bureau and third-party organizations to determine what lessons were
learned from the test and the implications on future overseas
enumeration efforts.
2004 Overseas Test Was Generally Implemented as Designed:
The Bureau's design for the 2004 overseas enumeration test was
generally implemented as planned and completed on schedule. The
Bureau's design had four key components: the mode of response, the
questionnaire designed specifically for Americans living overseas,
three test sites, and an outreach and promotion program designed to
communicate and educate Americans abroad that a test census was being
conducted. Table 1 describes each of these components in greater
detail.
Table 1: Key Components of Overseas Enumeration Were Generally
Implemented as Planned:
Key overseas test component: Census form response mode;
What the Bureau planned: Census questionnaire will be available in
paper format or via the Internet;
Generally implemented as planned? Yes;
GAO observations: Paper version of the census form was available at
various locations at the test sites; Questionnaire was available for
completion on the Internet; Paper version was also available in
Spanish in Mexico.
Key overseas test component: Census test questions;
What the Bureau planned: 2000 decennial short form questionnaire will
be modified for the overseas enumeration;
Generally implemented as planned? Yes;
GAO observations: Asked respondents to provide their passport numbers
and social security numbers to verify citizenship; Asked respondents
about their employment status--military, federal, or other; Asked for
information on everyone in the household even if the person was not a
U.S. citizen.
Key overseas test component: Test sites;
What the Bureau planned: Conduct test in three geographically diverse
areas with large American populations, and where administrative records
may be available to help verify results;
Generally implemented as planned? Yes;
GAO observations: Overseas test was implemented in three countries:
France, Kuwait, and Mexico.
Key overseas test component: Outreach/Promotion;
What the Bureau planned: Contract with a public relations firm to
develop a communications strategy to inform and motivate respondents
living in the selected countries to answer the census;
Generally implemented as planned? Yes;
GAO observations: Public relations firm hired to develop a
communication strategy; Strategy relied on public and private
organizations in each of the test sites; Organizations displayed
promotional materials about the test, communicated test census to
members, and distributed the paper census forms to American residents.
Source: GAO.
[End of table]
However, while the test was generally implemented as designed, our
earlier report pointed out several methodological limitations with the
design, such as not being able to calculate response rates because the
universe of Americans is unknown or not being able to measure the
quality of data because of the impracticality of developing an address
list. As we discuss later in this report, it is these methodological
limitations that impede the Bureau's ability to implement a successful
overseas enumeration.
Overseas Census Test Results Were Disappointing and Costly:
Although the 2004 overseas enumeration test ended in early July 2004
and the Bureau has just begun evaluating the results, the response
levels were poor, and very expensive to obtain on a per unit basis. The
response level to the overseas enumeration suggests that the current
approach to counting overseas Americans--a voluntary survey that relies
heavily on marketing to get people to participate--by itself cannot
secure a successful head count. Further, obtaining the additional
resources needed to produce substantially better results may not be
feasible, and still not yield data that are comparable in quality to
the stateside enumeration.
Response Levels Were Disappointing and Costly to Obtain:
The 5,390 responses the Bureau received for this test were far below
what the Bureau planned for when printing up materials and census
forms. While the Bureau ordered 520,000 paper forms for three test
sites, only 1,783 census forms were completed and returned. Of these,
35 were Spanish language forms that were made available in Mexico. The
remaining 3,607 responses were completed via the Internet. Table 2
below shows the number of census questionnaires that the Bureau printed
for each country and the number of responses they actually received in
both the paper format and via the Internet.
Table 2: Comparison of Responses Received for 2004 Overseas Census
Test:
Test sites: Mexico;
Number of forms printed for each test site: 430,000[A];
Number of responses by mode: Paper: 869[B];
Number of responses by mode: Internet: 1,130;
Total number of responses received: 1,999.
Test sites: France;
Number of forms printed for each test site: 75,000;
Number of responses by mode: Paper: 886;
Number of responses by mode: Internet: 2,219;
Total number of responses received: 3,105.
Test sites: Kuwait;
Number of forms printed for each test site: 15,000;
Number of responses by mode: Paper: 28;
Number of responses by mode: Internet: 258;
Total number of responses received: 286.
Total;
Number of forms printed for each test site: 520,000;
Number of responses by mode: Paper: 1,783;
Number of responses by mode: Internet: 3,607;
Total number of responses received: 5,390.
Source: U.S. Census Bureau.
[A] This includes 100,000 forms printed in Spanish.
[B] This includes 35 Spanish forms returned.
[End of table]
Because of the low response levels, in early April 2004, the Bureau
reversed its decision to not use paid advertising and in May 2004
initiated a paid advertising campaign in France and Mexico. This
included print and Internet ads in France and print and radio ads in
Mexico. See figure 2 for examples of the ads used in the paid
advertising campaign.
Figure 2: Census Bureau Ads Placed in the International Herald Tribune:
[See PDF for image]
[End of figure]
A Bureau official told us the ad campaign for the 2004 overseas test
cost about $206,000. This official said there were surplus funds
available in the project budget to use for this purpose due to lower
than expected processing and postage costs for the overseas test. While
the Bureau saw some increase in the number of responses after the paid
advertising campaign began, this official said the increase was slight.
Return on Investment for Overseas Enumeration is Low:
Not only were response levels low, they were extremely expensive to
obtain on a unit basis--roughly $1,450 for each returned questionnaire,
based on the $7.8 million the Bureau spent preparing for, implementing,
and evaluating the 2004 overseas test. In contrast, the unit cost of
the 2000 Census was about $56 per household. Although the two surveys
are not directly comparable because the 2000 Census costs covered
operations not used in the overseas test, the 2000 Census was still the
most expensive census in our nation's history.
Securing a Higher Return Rate Would Be an Enormous Challenge:
The main reason for the high unit cost is the low return rate. However,
significantly boosting participation levels may not be feasible. The
Bureau's experience in the 2000 Census highlights the level of effort
that was needed to raise public awareness about the census and get
people to complete their forms. For the 2000 decennial, the Bureau
spent $374 million on a comprehensive marketing, communications, and
partnership effort. The campaign consisted of a five-part strategy
conducted in three waves beginning in the fall of 1999 and continuing
past Census Day (April 1, 2000). The effort helped secure a 72-percent
return rate. Specific elements included television, radio, and other
mass media advertising; promotions and special events; and a census-in-
schools program. Thus, over a period of several months, the American
public was on the receiving end of a steady drumbeat of advertising
aimed at publicizing the census and motivating them to respond.
The Bureau also filled 594 full-time partnership specialist positions.
These individuals were responsible for mobilizing support for the
census on a grassroots basis by working with governmental entities,
private companies, and religious and social service groups, and other
organizations.
Replicating this level of effort on a worldwide basis would be
impractical, and still would not produce a complete count. Indeed, even
after the Bureau's aggressive marketing effort in 2000, it still had to
follow-up with about 42 million households that did not return their
census forms. Moreover, because there are no reliable figures on the
number of Americans residing overseas, the Bureau would not have a good
measure of the number of people that did not participate, or the
overall quality of the data.
Ensuring a Smooth Overseas Enumeration Could Stretch the Bureau's
Resources:
The Bureau's experience in conducting the 2004 overseas test
underscored the difficulties of administering a complex operation from
thousands of miles away. Not surprisingly, as with any operation this
complex, various challenges and unforeseen problems arose. While the
Bureau was able to resolve them, its ability to do so should there be a
full overseas enumeration as part of the 2010 Census, would be highly
questionable as far more resources would be required. This was
particularly evident in at least two areas: grappling with country-
specific issues and overseeing the contractor responsible for raising
public awareness of the census at the three test sites.
Country-specific Issues Created Implementation Problems:
The Bureau encountered a variety of implementation problems at each of
the test sites. In some cases the problems were known in advance, in
others, glitches developed at the last minute. Although such
difficulties are to be expected given the magnitude of the Bureau's
task, a key lesson learned from the test is that there would be no
economy of scale in ramping up to a full enumeration of Americans
abroad. In fact, just the opposite would be true. Because of the
inevitability of country-specific problems, rather than conducting a
single overseas count based on a standard set of rules and procedures
(as is the case with the stateside census), the Bureau might end up
administering what amounts to dozens of separate censuses--one for each
of the countries it enumerates--each with its own set of procedures
adapted to each country's unique requirements. The time and resources
required to do this would likely be overwhelming and detract from the
Bureau's stateside efforts.
For example, during the overseas test, the Bureau found that French
privacy laws restrict the collection of personal data such as race and
ethnic information. However, these data are collected as part of the
decennial census because they are key to implementing a number of civil
rights laws such as the Voting Rights Act.
Addressing France's privacy laws took a considerable amount of
negotiation between the two countries, and was ultimately resolved
after a formal agreement was developed. The Bureau issued and posted on
its Web site an advisory that informed Americans living in France that
it was not mandatory to respond to the questionnaire, the only
recipient of the collected data is the Census Bureau, the data will be
kept for one year, and the respondent has a right to access and correct
the data collected. The Bureau was able to collect race and ethnic
data--generally a prohibited practice without the respondents'
permission--only after it received special approval from a French
government agency. Initially, however, it looked as if the Bureau might
have to redesign the census form if it wanted to use it in France.
In Kuwait, delivery of the census materials was delayed by several
weeks at the beginning of the test because they were accidentally
addressed to the wrong contractor. Ultimately, the U.S. Embassy
stepped-in to accept the boxes so that the enumeration could proceed.
In Mexico, there was some initial confusion on the part of Mexican
postal workers as to whether they could accept the postage-paid
envelopes that the Bureau had provided to return the paper
questionnaires for processing in the United States.
Because of the small number of countries involved in the test, the
Bureau was able to address the various problems it encountered. Still,
the Bureau's experience indicates that it will be exceedingly difficult
to identify and resolve in advance all the various laws, rules,
societal factors, and a host of other potential glitches that could
affect a full overseas enumeration.
On-site Supervision of Contractor Was Problematic:
As noted previously, the Bureau hired a public relations firm to
develop a communications strategy to inform and motivate respondents
living in the test countries to complete the census. The firm's
responsibilities included identifying private companies, religious
institutions, service organizations, and other entities that have
contact with Americans abroad and could thus help publicize the census
test. Specific activities the organizations could perform included
displaying promotional materials and paper versions of the census
questionnaire, publishing information in a newsletter, and posting
information on their Web sites. Although the public relations firm
appeared to go to great lengths to enlist the participation of these
various entities--soliciting the support of hundreds of organizations
in the three countries--the test revealed the difficulties of
adequately overseeing a contractor operating in multiple sites
overseas.
For example, the public relations firm's tracking system indicated that
around 440 entities had agreed to perform one or more types of
promotional activities. However, our on-site inspections of several of
these organizations in Paris, France, and Guadalajara, Mexico, that had
agreed to display the census materials and/or distribute the
questionnaires, uncovered several glitches. Of the 36 organizations we
visited that were supposed to be displaying promotional literature, we
found the information was only available at 15. In those cases, as
shown in figure 3, the materials were generally displayed in prominent
locations, typically on a table with posters on a nearby wall.
Figure 3: Census Materials Were Prominently Displayed in Various
Locations in France and Mexico:
[See PDF for image]
[End of figure]
Five of these 15 organizations were also distributing the census
questionnaire, but the forms were not readily accessible.
However, at 21 sites we visited, we found various discrepancies between
what the public relations firm indicated had occurred, and what
actually took place. For example, while the firm's tracking system
indicated that questionnaires would be available at a restaurant and an
English-language bookstore in Guadalajara, none were available. In
fact, the owner of the bookstore told us that no one from the Census
Bureau or the public relations firm had contacted her about displaying
materials for the overseas test.
At the University of Guadalajara, although the tracking system
indicated that an official had been contacted about, and agreed to help
support the census test, that official told us no one had contacted
him. As a result, when boxes of census materials were delivered to his
school without any explanatory information, he did not know what to do
with them, and had to telephone the U.S. Consulate in Guadalajara to
figure out what they were for.
Likewise, in Paris, we went to several locations where the tracking
system indicated that census information would be available. None was.
In fact, at some of these sites, not only was there no information
about the census, but there was no indication that the organization we
were looking for was at the address we had from the database.
The results of the overseas test point to the difficulties of
overseeing the contractor's performance. As census materials were made
available at scores of locations across the three test countries, it
would have been impractical for the Bureau to inspect each site. The
difficulty of supervising contractors--and any field operation for that
matter--would only be magnified in a global enumeration.
The Design of the Overseas Census Lacks the Building Blocks of a
Successful Census:
The Bureau's experience in counting the nation's population for the
2000 and earlier censuses sheds light on some of the specific
operations and other elements that together form the building blocks of
a successful head count (see fig. 4).
Figure 4: Key Building Blocks of a Successful Census:
[See PDF for image]
[End of figure]
While performing these activities does not necessarily guarantee a
cost-effective headcount, not performing them makes a quality count far
less promising and puts the entire enterprise at risk. The current
approach to counting overseas Americans lacks these building blocks, as
most are infeasible to perform on an overseas population. Each is
discussed in greater detail below.
* Mandatory participation: Under federal law, all persons residing in
the United States regardless of citizenship status are required to
respond to the decennial census. By contrast, the overseas enumeration
test was conducted as a voluntary survey where participation was
optional. The Bureau has found that response rates to mandatory surveys
are higher than the response rates to voluntary surveys. This in turn
yields more complete data and helps hold down costs.
* Early agreement on design: Both Congress and the Bureau need to agree
on the fundamental design of an overseas census. Concurrence on the
design helps ensure adequate planning, testing and funding levels.
Conversely, the lack of an agreed-upon design raises the risk that
basic design elements might change in the years ahead, while the
opportunities to test those changes and integrate them with other
operations will diminish. Under the Bureau's current plans, after the
2006 test, the Bureau would have just one more opportunity to test its
prototype for an overseas enumeration--a dress rehearsal in 2008. Any
design changes after 2008 would not be tested in a real-world
environment. The design of the census is driven in large part by the
purposes for which the data will be used. Currently, no decisions have
been made on whether the overseas data will be used for purposes of
congressional apportionment, redistricting, allocating federal funds,
or other applications. Some applications, such as apportionment, would
require precise population counts and a very rigorous design that
parallels the stateside count. Other applications, however, could get
by with less precision and thus, a less stringent approach. As we noted
previously, Congress will need to decide whether or not to count
overseas Americans, and how the results should be used. The basis for
these determinations needs to be sound research on the cost, quality of
data, and logistical feasibility of the range of options for counting
this population group. Possibilities include counting Americans via a
separate survey, administrative records such as passport and voter
registration forms, and/or records maintained by other countries such
as published census records and work permits. The Bureau's initial
research has shown that each of these options has coverage, accuracy,
and accessibility issues, and some might introduce systemic biases into
the data. Far more extensive research would be needed to determine the
feasibility of these or other potential approaches. Once Congress knows
the tradeoffs of these various alternatives, it will be better
positioned to provide the Bureau with the guidance it needs to go
beyond research and conduct field tests of specific approaches. The
Bureau can conduct the research, or it can contract it out. Indeed, the
National Research Council of the National Academy of Sciences has
conducted a number of studies on the decennial census, including a
review of the 2000 Census and an examination of reengineering the 2010
Census.
* A complete and accurate address list: The cornerstone of a successful
census is a quality address list. For the stateside census, the Bureau
goes to great lengths to develop what is essentially an inventory of
all known living quarters in the United States, including sending
census workers to canvass every street in the nation to verify
addresses. The Bureau uses this information to deliver questionnaires,
follow up with nonrespondents, determine vacancies, and identify
households the Bureau may have missed or counted more than once.
Because it would be impractical to develop an accurate parallel address
list for overseas Americans, these operations would be impossible and
the quality of the data would suffer as a result.
* Ability to detect invalid returns: Ensuring the integrity of the
census data requires the Bureau to have a mechanism to screen out
invalid responses. Stateside, the Bureau does this by associating an
identification number on the questionnaire to a specific address in the
Bureau's address list, as well as by field verification. However, the
Bureau's current approach to counting overseas Americans is unable to
determine whether or not a respondent does in fact reside abroad. So
long as a respondent provides certain pieces of information on the
census questionnaire, it will be eligible for further processing. The
Bureau is unable to confirm the point of origin for questionnaires
completed on the Internet, and postmarks on a paper questionnaire only
tell the location from which a form was mailed, not the place of
residence of the respondent. The Bureau has acknowledged that ensuring
such validity might be all but impossible for any reasonable level of
effort and funding.
* Ability to follow up with non-respondents: Because participation in
the decennial census is mandatory, the Bureau sends enumerators to
those households that do not return their questionnaires. In cases
where household members cannot be contacted or refuse to answer all or
part of a census questionnaire, enumerators are to obtain data from
neighbors, a building manager, or other nonhousehold member presumed to
know about its residents. The Bureau also employs statistical
techniques to impute data when it lacks complete information on a
household. Thus, by the end of each decennial census, the Bureau has a
fairly exhaustive count of everyone in the nation. As noted above,
because the Bureau lacks an address list of overseas Americans, it is
unable to follow up with nonrespondents or impute information on
missing households. As a result, the Bureau will never be able to
obtain a complete count of overseas Americans.
* Cost model for estimating needed resources: The Bureau uses a cost
model and other baseline data to help it estimate the resources it
needs to conduct the stateside census. Key assumptions such as response
levels and workload are developed based on the Bureau's experience in
counting people decade after decade. However, the Bureau has only a
handful of data points with which to gauge the resources necessary for
an overseas census, and the tests it plans on conducting will only be
of limited value in modeling the costs of conducting a worldwide
enumeration in 2010. The lack of baseline data could cause the Bureau
to over-or underestimate the staffing, budget and other requirements of
an overseas count. For example, this was evident during the 2004
overseas test when the Bureau estimated it would need around 100,000
Spanish-language questionnaires for the Mexican test site. As only 35
Spanish-language questionnaires were returned, it is now clear that the
Bureau could have gotten by with printing far fewer questionnaires for
Mexico. However, the dilemma for the Bureau is that its experience in
the 2004 overseas test cannot be used to project the number of Spanish-
language questionnaires it would need for Mexico or other Spanish-
speaking countries in 2010. Similar problems would apply to efforts to
enumerate other countries.[Footnote 6]
* Targeted and aggressive marketing campaign: The key to raising public
awareness of the census is an intensive outreach and promotion
campaign. As noted previously, the Bureau's marketing efforts for the
2000 Census were far-reaching, and consisted of more than 250 ads in 17
languages that were part of an effort to reach every household,
including those in historically undercounted populations. Replicating
this level of effort on a global scale would be both difficult and
expensive, and the Bureau has no plans to do so.
* Field infrastructure to execute census and deal with problems: The
Bureau had a vast network of 12 regional offices and 511 local census
offices to implement various operations for the 2000 Census. This
decentralized structure enabled the Bureau to carry out a number of
activities to help ensure a more complete and accurate count, as well
as deal with problems when they arose. Moreover, local census offices
are an important source of intelligence on the various enumeration
obstacles the Bureau faces on the ground. For example, during the 2000
Census, the Bureau called on them to identify hard-to-count population
groups and other challenges, and to develop action plans to address
them. The absence of a field infrastructure for an overseas census
means that the Bureau would have to rely heavily on contractors to
conduct the enumeration, and manage the entire enterprise from its
headquarters in Suitland, Maryland.
* Ability to measure coverage and accuracy: Since 1980, the Bureau has
measured the quality of the decennial census using statistical methods
to estimate the magnitude of any errors. The Bureau reports these
estimates by specific ethnic, racial, and other groups. For
methodological reasons, similar estimates cannot be generated for an
overseas census. As a result, the quality of the overseas count, and
thus whether the numbers should be used for specific purposes, could
not be accurately determined.
Conclusions:
The 2004 test of the feasibility of an overseas enumeration was an
extremely valuable exercise in that it highlighted the numerous
obstacles to a cost-effective count of Americans abroad as an integral
part of the decennial census. Although more comprehensive results will
not be available until the Bureau completes its evaluation of the test
early next year, a key lesson learned is already clear: The current
approach to counting this population group--a voluntary survey that
largely relies on marketing to ensure a complete count--would be costly
and yield poor results. The tools and resources the Bureau has on hand
to enumerate overseas Americans are insufficient for overcoming the
inherent obstacles to a complete count, and it is unlikely that any
refinements to this basic design would produce substantially better
results, and certainly not on a par suitable for purposes of
congressional apportionment.
What's more, the Bureau already faces the difficult task of carrying
out a cost-effective stateside enumeration in 2010. Securing a
successful count of Americans in Vienna, Virginia, is challenging
enough; a complete count of Americans in Vienna, Austria--and in scores
of other countries around the globe--would only add to the difficulties
facing the Bureau as it looks toward the next national head count. As a
result, we believe that any further tests or planning activities
related to counting Americans overseas as part of the decennial census
would be an imprudent use of the Bureau's limited resources.
That said, to the extent that Congress desires better data on the
number and characteristics of Americans abroad for various policy-
making and other nonapportionment purposes that require less precision,
such information does not necessarily need to be collected as part of
the decennial census, and could, in fact, be acquired through a
separate survey or other means. To help inform congressional decision
making on this issue, including decisions on whether Americans should
be counted and how the data should be used, it will be important for
Congress to have the results of the Bureau's evaluation of the 2004
overseas census test. Equally important would be information on the
cost, quality of data, and logistical feasibility of counting Americans
abroad using alternatives to the decennial census. Once Congress knows
the tradeoffs of these various alternatives, it would be better
positioned to provide the Bureau with the direction it needs so that
the Bureau could then develop and test an approach that meets
congressional requirements at reasonable resource levels.
Matters for Congressional Consideration:
Given the obstacles to a cost-effective count of overseas Americans as
part of the decennial census and, more specifically, obtaining data
that is of sufficient quality to be used for congressional
apportionment, Congress may wish to consider eliminating funding for
any additional research, planning, and development activities related
to counting this population as part of the decennial headcount,
including funding for tests planned in 2006 and 2008. However, funding
for the evaluation of the 2004 test should continue as planned to help
inform congressional decision making.
Should Congress still desire better data on the number of overseas
Americans, in lieu of the method tested in 2004, Congress might wish to
consider authorizing and funding research on the feasibility of
counting Americans abroad using alternatives to the decennial census.
Recommendations for Executive Action:
To facilitate congressional decision making, we recommend that the
Secretary of Commerce ensure that the Bureau completes its evaluation
of the 2004 overseas census test as planned. Further, to the extent
that additional research is authorized and funded, the Bureau, in
consultation with Congress, should explore the feasibility of counting
overseas Americans using alternatives to the decennial census.
Potential options include:
* conducting a separate survey,
* examining how the design and archiving of various government agency
administrative records might need to be refined to facilitate a more
accurate count of overseas Americans, and:
* exchanging data with other countries' statistical agencies and
censuses, subject to applicable confidentiality and other provisions.
Consideration should also be given to whether the Bureau should conduct
this research on its own or whether it should be contracted out to the
National Academy of Sciences.
Agency Comments and Our Evaluation:
The Secretary of Commerce forwarded written comments from the U.S.
Census Bureau on a draft of this report on August 5, 2004, which are
reprinted in the appendix. The Bureau agreed with our conclusions and
recommendations. Furthermore, the Bureau noted, "should Congress
request and fund" further research on counting overseas Americans, it
would be equipped to do that research itself.
As agreed with your offices, unless you release its contents earlier,
we plan no further distribution of this report until 30 days from its
issue date. At that time we will send copies to other interested
congressional committees, the Secretary of Commerce, and the Director
of the U.S. Census Bureau. Copies will be made available to others upon
request. This report will also be available at no charge on GAO's Web
site at [Hyperlink, http://www.gao.gov].
If you or your staff have any questions concerning this report, please
contact me on (202) 512-6806 or by e-mail at
[Hyperlink, daltonp@gao.gov] or Robert Goldenkoff, Assistant Director,
at (202) 512-2757 or [Hyperlink, goldenkoffr@gao.gov]. Key
contributors to this report were Ellen Grady, Lisa Pearson, and
Timothy Wexler.
Signed by:
Patricia A. Dalton:
Director:
Strategic Issues:
[End of section]
Appendixes:
Appendix I: Comments from the Department of Commerce:
THE SECRETARY OF COMMERCE:
Washington, D.C. 20230:
Ms. Patricia A. Dalton:
Director, Strategic Issues:
U.S. Government Accountability Office:
Washington, DC 20548:
Dear Ms. Dalton:
The U.S. Department of Commerce appreciates the opportunity to comment
on the U.S. Government Accountability Office draft report entitled 2010
Census: Counting Americans Overseas as Part of the Decennial Census
Would Not Be Cost-Effective. The Department's comments on this report
are enclosed.
Sincerely,
Signed by:
Donald L. Evans:
Enclosure:
Comments from the U.S. Department of Commerce Regarding the U.S.
Government Accountability Office Draft Report Entitled 2010 Census:
Counting Americans Overseas as Part of the Decennial Census Would Not
Be Cost-Effective:
Thank you for providing us the opportunity to comment on your draft
report, 2010 Census: Counting Americans Overseas as Part of the
Decennial Census Would Not Be Cost-Effective.
Comments on Conclusions:
We agree with the draft report that the 2004 Overseas Census Test was
an extremely valuable exercise that highlighted the numerous obstacles
to a cost-effective count of Americans abroad as an integral part of
the decennial census.
Comments on Recommendations for Executive Action:
"To facilitate congressional decision-making, we recommend that the
Secretary of Commerce ensure that the Bureau completes its evaluation
of the 2004 overseas census test as planned."
The Census Bureau concurs with this recommendation.
"Further, to the extent that further research is authorized and funded,
the Bureau, in consultation with Congress, should research the
feasibility of counting overseas Americans using alternatives to the
decennial census. Potential options include ... a separate survey ...
administrative records, and ... [data exchange with other countries].
Consideration should also be given as to whether the Bureau should
conduct this research on its own or whether it should be contracted out
to the National Academy of Sciences."
The Census Bureau is equipped to conduct this research, should Congress
request and fund it.
[End of section]
(450315):
FOOTNOTES
[1] Only the 1970, 1990, and 2000 Censuses used counts of federally
affiliated personnel for purposes of apportioning Congress.
[2] Utah v. Evans, 143 F. Supp. 2d 1290 (D. Utah 2001), aff'd, Utah v.
Evans, 534 U.S. 1038 (2001).
[3] Congressional Research Service, House Apportionment: Could Census
Corrections Shift a House Seat?, RS21638 (Washington, D.C.: Oct. 8,
2003).
[4] GAO, 2010 Census: Overseas Enumeration Test Raises Need for Clear
Policy Direction, GAO-04-470 (Washington, D.C.: May 21, 2004).
[5] At this point, the Bureau does not have cost data beyond fiscal
year 2005. Therefore, it is unknown what the costs will be for
implementing and evaluating the 2006 overseas test or the 2008 overseas
dress rehearsal.
[6] The Bureau plans to destroy all unused questionnaires for the 2004
test.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
