Decennial Census
Methods for Collecting and Reporting Data on the Homeless and Others without Conventional Housing Need Refinement Gao ID: GAO-03-227 January 17, 2003The Bureau of the Census partnered with local governments, advocacy groups, and other organizations to help it enumerate people without conventional housing. Counting this population--which includes shelter residents and the homeless--has been a longstanding challenge for the Bureau. A number of organizations put substantial resources into an operation the Bureau called Service-Based Enumeration. In return, some expected the Bureau to provide data that would help them plan and deliver employment, health, and other services. However, the Bureau did not release the data as planned, which raised questions about the Bureau's decision-making on data quality issues. In response to a congressional request, GAO examined the Bureau's decision-making process behind its change in plans.
The Bureau's original plan for releasing Service-Based Enumeration data was outlined in an April 1999 internal memorandum that called for the separate release of data on people counted at "emergency and transitional shelters." The Bureau planned to combine other components of Service-Based Enumeration, including people counted at soup kitchens, regularly scheduled mobile food vans, and certain outdoor locations, into a single category. Driving the Bureau's decision was its experience during the 1990 Census when it released separate counts of people found at shelters, on the street, and similar locations that proved to be incomplete. The Bureau also tried to ensure that the Service-Based Enumeration figures could not be used as a "homeless" count, because it was not designed to provide a specific count of the homeless. Instead, the operation was part of a larger effort to count people without conventional housing. In January 2001, the Bureau changed its earlier decision because a statistical procedure used to refine the emergency and transitional shelter data proved to be unreliable, which lowered the quality of the data. In response, the Bureau combined the shelter data with a category called "other non-institutional group quarters," a category that also includes data on people enumerated in several other group locations such as facilities for victims of natural disasters. In the fall of 2001, the Bureau produced a heavily qualified special report on the shelter data. A key cause of the Bureau's shifting position on reporting these data appears to be its lack of well documented, transparent, clearly defined, and consistently applied guidelines on the minimum quality necessary for releasing data. Had these guidelines been in place at the time of the census, the Bureau could have been better positioned to make an objective decision on releasing these figures. Additionally, the Bureau could have used the guidance to explain to data users the reasons for the decision, eliminating any appearance of censorship and arbitrariness. Because the Bureau did not always adequately communicate its plans for releasing the data, expectation gaps developed between the Bureau and entities that helped with Service-Based Enumeration.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-227, Decennial Census: Methods for Collecting and Reporting Data on the Homeless and Others without Conventional Housing Need Refinement
This is the accessible text file for GAO report number GAO-03-227
entitled 'Decennial Census: Methods for Collecting and Reporting
Data on the Homeless and Others without Conventional Housing Need
Refinement' which was released on January 17, 2003.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States General Accounting Office:
GAO:
Report to Congressional Requesters:
January 2003:
Decennial Census:
Methods for Collecting and Reporting Data on the Homeless and Others
without Conventional Housing Need Refinement:
GAO-03-227:
GAO Highlights:
Highlights of GAO-03-227, a report to Congressional Requesters.
Why GAO Did This Study:
The Bureau of the Census partnered with local governments, advocacy
groups, and other organizations to help it enumerate people without
conventional housing. Counting this population”which includes shelter
residents and the homeless”has been a longstanding challenge for the
Bureau. A number of organizations put substantial resources into an
operation the Bureau called Service-Based Enumeration. In return, some
expected the Bureau to provide data that would help them plan and
deliver employment, health, and other services. However, the Bureau did
not release the data as planned, which raised questions about the
Bureau‘s decisionmaking on data quality issues. In response to a
congressional request, GAO examined the Bureau‘s decision-making process
behind its change in plans.
What GAO Found:
The Bureau's original plan for releasing Service-Based Enumeration data
was outlined in an April 1999 internal memorandum that called for the
separate release of data on people counted at "emergency and
transitional shelters." The Bureau planned to combine other components
of Service-Based Enumeration, including people counted at soup
kitchens, regularly scheduled mobile food vans, and certain outdoor
locations, into a single category. Driving the Bureau's decision was
its experience during the 1990 Census when it released separate counts
of people found at shelters, on the street, and similar locations that
proved to be incomplete. The Bureau also tried to ensure that the
Service-Based Enumeration figures could not be used as a "homeless"
count, because it was not designed to provide a specific count of the
homeless. Instead, the operation was part of a larger effort to count
people without conventional housing.
Figure: The Homeless Are Hard to Enumerate:
{See PDF for image]
Photograph of a homeless person on a park bench.
Source: U.S. Bureau of the Census.
[End of figure]
In January 2001, the Bureau changed its earlier decision because a
statistical procedure used to refine the emergency and transitional
shelter data proved to be unreliable, which lowered the quality of the
data. In response, the Bureau combined the shelter data with a category
called "other noninstitutional group quarters," a category that also
includes data on people enumerated in several other group locations
such as facilities for victims of natural disasters. In the fall of
2001, the Bureau produced a heavily qualified special report on the
shelter data. A key cause of the Bureau's shifting position on
reporting these data appears to be its lack of well documented,
transparent, clearly defined, and consistently applied guidelines on the
minimum quality necessary for releasing data. Had these guidelines been
in place at the time of the census, the Bureau could have been better
positioned to make an objective decision on releasing these figures.
Additionally, the Bureau could have used the guidance to explain to data
users the reasons for the decision, eliminating any appearance of
censorship and arbitrariness. Because the Bureau did not always
adequately communicate its plans for releasing the data, expectation
gaps developed between the Bureau and entities that helped with Service-
Based Enumeration.
What GAO Recommends:
The Secretary of Commerce should direct the Bureau to (1) properly test
and evaluate procedures for counting people without conventional
housing; (2) develop guidelines for decisions on the level of quality
needed to release data to the public, how to characterize any
limitations, and when it is acceptable to suppress data; and (3) ensure
that plans for releasing data are clearly communicated to data users.
The Bureau agreed with GAO‘s recommendations, but took issue with our
findings on the adequacy of its data quality guidelines.
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-227].
To view the full report, including the scope and methodology, click on
the link above. For more information, contact Patricia A. Dalton at
(202) 512-6806.
[End of section]
Contents:
Letter:
Results In Brief:
Background:
Scope and Methodology:
The Bureau of the Census Twice Changed Plans for Reporting Service-
Based Enumeration Data:
Census Bureau Had Few Documented Guidelines Governing the Release of
Census Data:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix:
Appendix I: Comments from the Secretary of Commerce:
Related GAO Products on the Results of the 2000 Census and Lessons
Learned for a More Effective Census in 2010:
Figure:
Figure 1: The Bureau Changed Its Original Plan to Release Emergency and
Transitional Shelter Data and Combined Them with Other Noninstitutional
Group Quarters Data:
[End of section]
United States General Accounting Office:
Washington, D.C. 20548:
January 17, 2003:
The Honorable Henry A. Waxman:
Ranking Minority Member:
Committee on Government Reform:
House of Representatives:
The Honorable Danny K. Davis:
Ranking Minority Member:
Subcommittee on Civil Service, Census and Agency Organization:
Committee on Government Reform:
House of Representatives:
The Honorable Carolyn B. Maloney:
The Honorable Dennis J. Kucinich:
The Honorable Wm. Lacy Clay:
House of Representatives:
For the 2000 Census, the Bureau of the Census employed several
initiatives to help ensure a complete and accurate count of people
without conventional housing. Enumerating this segment of the
population, which contains, among others, people referred to as
’homeless,“ has been an ongoing problem for the Bureau. In one
initiative, known as Service-Based Enumeration, census enumerators
attempted to count these individuals at emergency and transitional
shelters, soup kitchens, regularly scheduled mobile food vans, as well
as at what the Bureau calls ’targeted nonsheltered outdoor locations“
(TNSOL) for people living on the street at targeted locations who do
not use services. To help locate and count people, the Bureau partnered
with organizations providing services to the homeless and local
governments, some of which put substantial resources into their
efforts. In return, some of these organizations expected the Bureau to
provide data that would help them plan and deliver health, employment,
and other services directed toward this population.
However, in its review of the emergency and transitional shelter data,
the Bureau identified serious concerns with the quality of the data and
concluded that the data should not be released without explanation of
their extensive limitations and caveats. As a result, the Bureau
decided not to separately report the emergency and transitional shelter
data in the initial release of summary files as originally planned.
Instead, the Bureau combined the emergency and transitional shelter
data with a category called ’other non-institutional group quarters.“
At the census tract level (small statistical subdivisions of counties)
this category included soup kitchens, regularly scheduled mobile food
vans, shelters for victims of domestic violence, residential care
facilities providing protective oversight, staff quarters including
those for nurses and interns at military and general hospitals, and
living quarters for victims of natural disasters. At the block level,
in addition to the categories listed above, ’other non-institutional
group quarters“ included group homes, religious group quarters, other
nonhousehold living situations, and workers‘ dormitories. Aggregating
the shelter numbers with these other data raised concerns among some
data users that the Bureau was suppressing the results.
You asked us to examine the Bureau‘s decision-making process behind its
change in plans. As agreed with your offices, this report examines (1)
the Bureau‘s plans for reporting the results of Service-Based
Enumeration and its reasons for changing those plans and (2) the
Bureau‘s protocols for releasing data.
Members of the Congress also raised concerns about the quality of
Hispanic subgroup data and asked us to review the Bureau‘s decision-
making process for collecting and reporting ethnicity information. The
results of that study are included in a companion report. [Footnote 1]
Both reports are part of our ongoing series on the lessons learned from
the 2000 Census that can help inform the planning effort for 2010. (See
the Related GAO Products section for a list of reports issued to date
on census issues.)
Results in Brief:
The Bureau‘s original plan for disseminating Service-Based Enumeration
data was outlined in an April 1999 internal memorandum that called for
the separate release of data on ’emergency and transitional shelters,“
but did not specify why the Bureau was not separately releasing data on
the other locations enumerated during the Service-Based Enumeration”soup
kitchens, regularly scheduled mobile food vans, shelters for victims of
domestic violence, and targeted nonsheltered outdoor locations. The
Bureau‘s plan reflected its experience during the 1990 Census when it
released separate counts of people found at emergency shelters, street
locations, and similar locations. However, those counts proved to be
incomplete. The Bureau indicated from the beginning that these 1990
counts could not be added together to produce a count of the homeless
population. Despite the Bureau‘s warnings to the contrary, the data were
often misinterpreted as a ’homeless“ count. Thus, in developing its data
release plans, the Bureau took steps to ensure that the Service-Based
Enumeration figures could not be added together and used as a homeless
count.
In January 2001, the Bureau changed its earlier decision to include the
data on emergency and transitional shelters in one of its early data
releases because a procedure used to refine the Service-Based
Enumeration data proved to be unreliable. Although the Bureau had
tested the procedure earlier in the decade, because of methodological
limitations the test did not reveal any flaws. However, because
problems with the procedure surfaced during the review of 2000 Census
operations, the Bureau decided to combine the emergency and
transitional shelter data with the ’other noninstitutional group
quarters“ category that also includes data on people enumerated in the
other categories of the Service-Based Enumeration and in several other
group locations, such as facilities for victims of natural disasters.
As a result of this decision, the Bureau did not separately report any
data from the Service-Based Enumeration in its initial release of
Census 2000 data. These were the only data with separate reporting
categories that the Bureau decided to collapse into another category.
In the fall of 2001, the Bureau produced a special report on the
emergency and transitional shelter data”including most of the same data
that the Bureau earlier stated it could not release because of quality
concerns. This report did not include data on targeted nonsheltered
outdoor locations, or on soup kitchens and mobile food vans. The Bureau
added a lengthy discussion of the limitations of the data and
emphasized that they should not be interpreted as a count of the
homeless population.
Although the Bureau worked closely with a number of government entities,
advocacy groups, and other organizations to conduct Service-Based
Enumeration, reconciling its often competing data needs proved
challenging. Compounding the Bureau‘s difficulties, expectation gaps
developed between these entities and the Bureau because the Bureau did
not always clearly and consistently communicate its plans.
A key cause of the Bureau‘s shifting position on reporting the Service-
Based Enumeration data appears to be its lack of clear, documented, and
consistently applied guidelines governing the release of data from the
2000 Census. Had these guidelines been in place at the time of the
census, they could have helped Bureau managers decide whether to
release the Service-Based Enumeration data and how to characterize
these data. Additionally, the Bureau could use the guidelines to defend
its decisions once they were made, thus helping to ensure that the
Bureau‘s decisions both are, and appear to be, completely objective.
To ensure that the 2010 Census will provide data users with more
complete, accurate, and useful information on people without
conventional housing, we recommend that the Secretary of Commerce
direct the Bureau of the Census to ensure that the procedures for
enumerating and estimating segments of the population without
conventional housing are properly tested and evaluated under conditions
as similar to the census as possible. In addition, the Bureau should
develop clearly documented, transparent, and consistently applied
agencywide guidelines for releasing all census data to the public and
ensure that plans for releasing data are clearly and consistently
communicated to the public.
The Secretary of Commerce forwarded written comments from the Bureau
of the Census on a draft of this report (see app. I). The Bureau agreed
with our recommendations and is taking steps to implement them, but took
exception to our findings concerning the adequacy of its data quality
guidelines and communication with the public.
Background:
The procedures the Bureau used during the 1990 Census to count people
without conventional housing had limitations that resulted in incomplete
data. [Footnote 2] To address these limitations and help improve the
quality of the data, the Bureau used a procedure for the 2000 Census
called Service-Based Enumeration that attempted to count people where
they receive services such as emergency shelters, soup kitchens, and
regularly scheduled mobile food vans. Service-Based Enumeration also
counted people in targeted nonsheltered outdoor locations such as
encampments beneath bridges. The operation occurred from March 27
through March 29, 2000. [Footnote 3]
According to Bureau officials, Service-Based Enumeration was not
designed, and was never intended, to provide a specific count of
homeless persons. Instead, the operation was part of a larger effort to
count people without conventional housing, including people in
’institutional group quarters“ such as correctional facilities, nursing
homes, and mental hospitals, and ’non-institutional group quarters“
such as college dormitories, military quarters, and group homes.
Service-Based Enumeration counted people in specific categories of
noninstitutional group quarters.
To help ensure a complete count of people without conventional housing,
the Bureau partnered with local governments and community advocacy
groups to obtain lists of service locations and to assist with the
enumeration. [Footnote 4] In some cases, the Bureau hired clients of
the advocacy groups and other people trusted by the homeless to conduct
Service-Based Enumeration. For example in Atlanta, an advocacy group
for homeless veterans helped the Bureau employ homeless veterans to
improve the count of this population. Local governments helped the
Bureau as well, often investing considerable resources. For example,
Los Angeles paid to keep its city-run shelters open on the night they
were enumerated so that people using their services could be counted.
Scope and Methodology:
To address your concerns about the Bureau‘s dissemination of data on
persons without conventional housing, we agreed to examine (1) the
Bureau‘s plans for reporting the results of Service-Based Enumeration
and its reasons for changing those plans and (2) the Bureau‘s protocols
for releasing data. To accomplish these objectives, we interviewed key
Bureau officials and reviewed relevant Bureau documents and data such as
operational plans, decision memorandums, and the Bureau‘s partnership
program evaluation.
In order to obtain the perspective of data users, partners, and
stakeholders, we conducted in-person and telephone interviews with
homeless advocates, local government officials, and representatives of
public service agencies in New York City, Los Angeles, Cleveland,
Atlanta, and Washington, D.C. These cities had large numbers of people
without conventional housing and they were actively involved with the
Bureau during the 2000 Census. The organizations we contacted also
provided relevant documentation, such as comprehensive file documents
relating to partnership activities.
In addition to the above locations, we did our audit work at Bureau
headquarters in Suitland, Maryland. Our audit work was conducted from
April 2002 through September 2002 in accordance with generally accepted
government auditing standards.
We requested comments on a draft of this report from the Secretary of
Commerce. On November 21, 2002, the Secretary forwarded the Bureau‘s
written comments on the draft (see app. I). We address these comments at
the end of this report.
The Bureau of the Census Twice Changed Plans for Reporting Service-Based
Enumeration Data:
Under the Bureau‘s original plan for releasing Service-Based Enumeration
data in Summary File-1 (SF-1), [Footnote 5] the emergency and
transitional shelter count was one of several categories of
noninstitutional group quarters data that were to be reported
separately. Other people counted in the Service-Based Enumeration,
including people counted at targeted nonsheltered outdoor locations,
soup kitchens, and regularly scheduled mobile food vans, were to be
combined and reported under the category ’other noninstitutional group
quarters.“ This category also included residential care facilities
providing protective oversight, shelters against domestic violence,
staff dormitories for nurses and interns at military and general
hospitals, and living quarters for victims of natural disasters.
This decision was documented in an April 1999 internal memorandum from
the Bureau‘s Assistant Division Chief for Special Population Statistics
to the Assistant Division Chief for Census Programs. The Service-Based
Enumeration operation took place a year later, in March 2000. The April
1999 plan was in large part a reaction to the challenges the Bureau
faced counting the emergency shelter and street population during the
1990 Census. Although the Bureau disseminated separate counts of people
found at emergency shelters, preidentified street locations, and similar
sites, the counts proved to be incomplete. [Footnote 6] Moreover, the
Bureau stated in its October 2001 report that despite its warnings to
the contrary, the data were sometimes misinterpreted as a ’homeless“
count.
The October report does not offer an example of this, but the
misinterpretation clearly played a role in a lawsuit against the
Bureau. [Footnote 7] As a result, when designing the 2000 census, the
Bureau attempted to both improve the count and take precautions to
ensure that the Service-Based Enumeration count would not be
misconstrued as a count of the homeless.
The Bureau‘s data dissemination plans took into account the
recommendations of the Commerce Secretary‘s 2000 Census Advisory
Committee, a panel that included representatives of advocacy and other
groups (including representatives from organizations that represent
local governments) that met periodically to review the Bureau‘s plans.
The homeless population was represented by the National Coalition for
the Homeless”an advocacy group that coordinates a network of 300 state
and local housing and homeless organizations. In its January 1999 final
report, the Census 2000 Advisory Committee recommended that special
attention be paid to tabulating the results of Service-Based
Enumeration and targeted outdoor enumerations so that they could not be
aggregated and used as a homeless count.
The Bureau Changed Its Dissemination Plans Because of Data Quality
Concerns:
In January 2001, 5 months before the SF-1 release, the Bureau reversed
its April 1999 decision to release emergency and transitional shelter
data separately because of ’data quality concerns.“ Instead, as shown
in figure 1, the Bureau planned to combine the emergency and
transitional shelter data with the ’other non-institutional group
quarters.“ This category contained data on a variety of living
arrangements including facilities for natural disaster victims. The
Bureau‘s decision was contained in an internal Bureau memorandum from
the Chief of the Population Division to the Chief of the Decennial
Systems and Contracts Management Office. Bureau officials told us that
the decision to exclude a separate emergency and transitional shelter
count in SF-1 was made between December 2000 and January 2001, by the
Director of the Decennial Census with input from the Associate Director
Decennial Census, the Population Division, the Associate Director for
Demographic Programs, the Decennial Management Division, and the
Decennial Statistical Studies Division. [Footnote 8]
Figure 1: The Bureau Changed Its Original Plan to Release Emergency and
Transitional Shelter Data and Combined Them with Other Noninstitutional
Group Quarters Data:
[See PDF for image]
This figure contains illustrations of both plans, as follows:
Original plan (April 1999):
Group quarters:
Institutionalized population reporting categories:
* Correctional institutions;
* Nursing homes;
* Hospitals;
* Juvenile institutions.
Non-Institutionalized Population reporting categories:
* College dorms;
* Military quarters;
* Group homes;
* Religious group quarters;
* Worker dormitories;
* Crews of maritime vessels;
* Emergency and transitional shelters;
* Other noninstitutional group quarters.
The other noninstitutional group quarters categories are:
- Soup kitchens and regularly scheduled food vans;
- Targeted nonsheltered outdoor locations;
- Residential facilities "providing protective oversight";
- Living quarters for victims of natural disasters;
- Staff residents of institutions;
- Shelters for victims of domestic violence.
Revised plan (January 2001):
Institutionalized population reporting categories:
* Correctional institutions;
* Nursing homes;
* Hospitals;
* Juvenile institutions.
Non-Institutionalized Population reporting categories:
* College dorms;
* Military quarters;
* Group homes;
* Religious group quarters;
* Worker dormitories;
* Crews of maritime vessels;
* Other noninstitutional group quarters.
The other noninstitutional group quarters categories are:
- Emergency and transitional shelters (this was moved from the original
plan);
- Soup kitchens and regularly scheduled food vans;
- Targeted nonsheltered outdoor locations;
- Residential facilities "providing protective oversight";
- Living quarters for victims of natural disasters;
- Staff residents of institutions;
- Shelters for victims of domestic violence.
Source: GAO depiction based on Bureau of the Census data.
[End of figure]
According to Bureau officials, their concerns focused on the accuracy
of a new statistical procedure called ’multiplicity estimation“ that
adjusted the number counted to better reflect the number of actual
shelter users. Because Service-Based Enumeration only counted people
who were at these facilities on the day of enumeration, the Bureau
intended to use multiplicity estimation to calculate the number of
people who used these facilities but were not present during Service-
Based Enumeration. The multiplicity estimation procedure was based on
information from those who were counted and on the number of times they
used the service facilities in the prior week. An estimate of people
not counted on the day of enumeration was added to the count of people
who were. According to the Bureau, the multiplicity estimates tested
well during the 1998 dress rehearsal for the 2000 Census possibly
because the three rehearsal sites did not offer large enough sample
sizes of the appropriate populations to adequately test this procedure.
[Footnote 9] However, during the 2000 Census the Bureau found that a
census question pertaining to facility usage upon which the
multiplicity estimates were based had a low response rate. Moreover,
the Bureau found that respondents, particularly in shelters, did not
answer the question accurately. Due to data quality concerns, the
Bureau decided not to use multiplicity estimation to adjust the data and
consequently decided not to report the data separately.
Bureau officials said they did not announce the change in plans because
they were still evaluating the problems with the data. It was not until
June 2000 that the Bureau began recalculating the data and making a
final decision on which categories to aggregate. Ultimately, the Bureau
did not report any of the Service-Based Enumeration data separately in
SF-1. Emergency and transitional shelter data were the only data that
were to be released in SF-1 under separate reporting categories that
the Bureau decided to combine with another category.
The Bureau Produced a Special Report in October 2001 on the Emergency
and Transitional Shelter Population:
The release of the SF-1 data in June 2001 produced public discussion in
the press, among census partners, and in the Congress about the Bureau‘s
decision to not separately release Service-Based Enumeration data. In a
briefing for staff of the House Committee on Governmental Affairs, the
Associate Director of the Decennial Census announced that the Bureau
planned to produce a separate report on the emergency and transitional
shelter data. In October 2001, the Bureau issued a special report,
entitled Emergency and Transitional Shelter Population: 2000. This
report separately identified emergency and transitional shelter data
for various levels of geography down to the census tract level with 100
or more people in emergency and transitional shelters. The report did
not include data for the populations in targeted nonsheltered outdoor
locations, soup kitchens, regularly scheduled mobile food vans, and
shelters for domestic violence. The 17-page report contains an
extensive discussion on the limitations of the data. For example, the
Bureau noted that the data in the report should not be construed as a
count of people without conventional housing. Moreover, the emergency
and transitional shelter data at the census tract level are not in the
hard copy, but rather in the Internet version of the report. [Footnote
10] The Bureau stated that all Census 2000 data at the tract level are
available on the Internet and are not available in printed reports.
The October report contains most of the same data that were to be
released under the April 1999 dissemination plan for SF-1. The Bureau
asserted that the data quality concerns with the emergency and
transitional shelter data (cited when it changed the plan to release
these data in SF-1) required that the data be presented in a manner
that allowed the Bureau to clearly outline the data‘s limitations. The
October 2001 report contained an extended discussion of these
limitations.
The October 2001 report also identified reasons the Bureau did not (and
never planned to) separately release data on people counted at targeted
nonsheltered outdoor locations, soup kitchens, regularly scheduled
mobile food vans, and shelters for victims of domestic violence,
including the following.
* People without conventional housing who were at outside locations
other than the targeted nonsheltered outdoor locations identified for
the census were not included in the TNSOL operation.
* For the purposes of the TNSOL operation, the definition of ’outdoor“
excluded both mobile and transient locations used by people
experiencing homelessness as well as abandoned buildings.
* The option was given to the individuals found at soup kitchens and
regularly scheduled mobile food vans to select ’usual home elsewhere.“
For example, if an individual enumerated at a soup kitchen listed a
usual home elsewhere, then that person was tabulated at their usual
residence and not at the service location. Therefore, the data on this
population would not reflect a true count of the individuals using
these services.
Prior to publicly releasing the October report, the Bureau asked two
representatives from the National Coalition for the Homeless to review a
draft of the portion of the report that described the limitations of
the data. The National Coalition for the Homeless commented extensively
on the section containing the caveats and limitations in order to
strengthen the report. A member of the Board of Directors for the
National Coalition for the Homeless told us that he provided this
feedback both as an academician and a stakeholder. Bureau officials
stated that because of its position on the Bureau‘s Census Advisory
Committee, the National Coalition for the Homeless was the only
advocacy group that reviewed any portion of the October 2001 report
prior to its publication.
Meeting Data Users‘ Needs Proved Challenging:
The controversy surrounding the release of the combined Service-Based
Enumeration data highlights the challenges the Bureau faced in 2000
trying to meet the needs of various data users and the work the Bureau
still needs to do when planning for the 2010 Census to better reconcile
those needs. For example, several organizations we contacted favored
the separate release of the Service-Based Enumeration data categories.
Indeed, local government officials we talked to in New York City
believed that the data would help with grant applications, projections
about future service needs, and determining their success in getting
people off the streets and into shelters. The Executive Director of the
Northeast Ohio Coalition for the Homeless stated that the city of
Cleveland does not do its own count of this population and, therefore,
the Bureau numbers are the only ones available on this segment of the
population. Los Angeles city officials wanted the Service-Based
Enumeration data so they could better target their services and, like
Cleveland, Los Angeles did not have its own data. Several of these
entities stated that the potential misuse of data was not a valid
reason for not separately releasing data.
In addition, the majority of the organizations we contacted partnered
with the Bureau anticipating that they would be able to use the Service-
Based Enumeration data to evaluate whether improvements were made in
enumerating local populations without conventional housing in 2000
compared to 1990. The Assistant City Attorney of Los Angeles estimated
that Los Angeles spent about $300,000 on the effort to improve the
count of Los Angeles‘s people without conventional housing. For
example, as part of an extensive effort to help the Bureau develop a
list of targeted nonsheltered outdoor locations, the city provided
senior Bureau staff with a helicopter tour over some outdoor locations
where people without conventional housing lived. The Assistant City
Attorney of Los Angeles stated that she believed the city would get the
targeted nonsheltered outdoor locations data that they helped collect
and wanted to review. In addition, because of the Bureau‘s focus on
counting people at shelters, the city kept shelters open on the night
of the enumeration at its own expense even though shelters in Los
Angeles typically do not have many people during warm weather. Los
Angeles expected to have detailed data to use to evaluate the
effectiveness of its resource allocation.
However, the National Coalition for the Homeless and other advocates of
the homeless opposed the separate release of any of the Service-Based
Enumeration data. They were concerned that these data could be misused
as a count of the homeless population and lead to flawed decision-making
by policymakers.
Ultimately, the Bureau left a number of data users unsatisfied. Those
who wanted the Service-Based Enumeration categories released separately
did not feel the Bureau met their expectations with the data released
in SF-1 or with the release of the October report. Users who opposed
the separate release of the data and were pleased that SF-1 combined
the Service-Based Enumeration components with other data were
displeased that the October 2001 report was released.
The difficulties the Bureau experienced trying to reconcile the
competing needs and interests of data users illustrates the importance
of effective communication between the Bureau and its key data users
and partners to ensure no expectation gaps develop. More than just a
good business practice, federal internal control standards require
agencies to have effective external communications with groups that can
have a serious impact on programs, projects, operations, and other
activities. [Footnote 11]
However, our conversations with several Bureau partners and our review
of Bureau documents suggest that communications were sometimes vague
and insufficient. For example, although the April 1999 memorandum that
outlined the Bureau‘s initial data dissemination plans was written a
year before the 2000 Census, this information may not have been
effectively communicated to the Bureau‘s partners. Indeed, at a Capitol
Hill briefing on this topic in June 2001, Bureau officials themselves
acknowledged that they did not do a good job of communicating on this
issue. Some of the partners we spoke to indicated that had they known
earlier about the Bureau‘s plans to limit the release of Service-Based
Enumeration data they might have focused their resources on different
census operations. Further, our review of Bureau documents indicated
that the information on the ’official plan“ for the release of the
different Service-Based Enumeration categories of data was limited and
inconsistent. Some partners stated that they did not know that the
Bureau never intended to report the targeted nonsheltered outdoor
location data.
Although the Bureau made numerous presentations on Service-Based
Enumeration that emphasized there would be no count of the homeless,
the Bureau provided little detail on how components of Service-Based
Enumeration would actually be presented. In the absence of clear
communication from the Bureau, partners developed their own
expectations of what would be released. Several of the local officials
and advocates that we spoke to expected that the data would be released
in the same detail as it was in 1990, because they were not told
otherwise. For example, a Los Angeles government official said that the
Bureau stated it would not provide a homeless count in 1990, but it
still released the street count data separately. By focusing resources
on counting specific categories of the population, the Bureau may have
created expectations that there would be a count of that population.
Census Bureau Had Few Documented Guidelines Governing the Release of
Census Data:
A cause of the Bureau‘s shifting position on reporting the components of
Service-Based Enumeration appears to be its lack of documented, clear,
transparent, and consistently applied guidelines governing the release
of data from the 2000 Census. Except for some guidance aimed at
protecting the confidentiality of census records, the Bureau had few
written guidelines on the level of quality needed to release data to
the public. Had these guidelines been in place during the decennial
census, they could have informed the Bureau‘s decision on whether to
release the Service-Based Enumeration data, how to characterize these
data, and help defend the decision after it was made. Such guidelines
could also provide a basis ahead of time for expectations about the
conditions under which data will or will not be released.
Although Bureau officials emphasized that the Bureau has a long
tradition of high standards and procedures that yield quality data (to
its credit, the Bureau‘s quality assurance practices identified the
problem with the multiplicity estimator), the officials acknowledged
that these standards were primarily part of the agency‘s institutional
knowledge. The written guidance that did exist appeared to be vague and
insufficient for making consistent decisions on the quality thresholds
needed for releasing data to the public, and the circumstances under
which it might be appropriate to suppress certain data.
According to the Bureau‘s Associate Director for Methodology and
Standards, a technical paper issued in 1974 and revised in 1987
contained the Bureau‘s only written guidelines for discussion and
presentation of errors in data. This paper noted that, ’estimates for
individual cells of a published table should not be suppressed solely
because they are subject to large sampling errors or large nonsampling
variances, provided users are given adequate caution of the lack of
reliability of the data. On the other hand, data known to have very
serious bias may be suppressed.“ [Footnote 12]
A Newly Created Bureau Program Could Provide Guidelines for Releasing
Data:
In 2000, the Bureau initiated a new quality assurance program to
document Bureau-wide protocols designed to ensure the quality of data
collected and disseminated by the Bureau. The Bureau‘s Methodology and
Standards Council is charged with setting statistical and survey
quality standards and guidelines for Bureau surveys and censuses. In
support of this role, the council has established a quality framework
in which the demographic, economic, and decennial areas can share and
support common principles, standards, and guidelines. The quality
framework covers eight unique areas, one of which is dissemination.
Because this Bureau program is in its initial stages, we could not
evaluate it. However, Bureau officials believe that the program is a
significant first step in addressing the lack of agencywide written
guidelines for releasing data. The initiative appears to be consistent
with Office of Management and Budget guidelines issued in February 2002
requiring federal agencies to issue their own guidance for ensuring and
maximizing the quality, objectivity, utility, and integrity of
information disseminated by the agency. [Footnote 13] As the Bureau
develops its guidelines, it will be important that they be well
documented, transparent, clearly defined, and consistently applied.
Conclusions:
Although Service-Based Enumeration was designed to address the
challenges the Bureau encountered during the 1990 Census in obtaining a
complete count of people without conventional housing, the Bureau‘s
experience during the 2000 Census suggests that tallying this
population group remains problematic. Moreover, the Bureau‘s
difficulties were compounded by its shifting position on how to report
the data once they were collected. A number of government, community,
and advocacy organizations helped the Bureau enumerate this population
group. However, the Bureau, by first planning to release the data one
way, then changing the decision, and ultimately releasing the data
anyway”all for reasons that were not clearly articulated to the
Bureau‘s stakeholders”raised questions about the Bureau‘s decision-
making on data quality issues. As noted at the beginning of this
report, related questions have also been raised about how the Bureau
collected and reported data on Hispanic subgroups. To the extent
similar incidents occur in the future, they could undermine public
confidence in the accuracy and credibility of Bureau data.
Thus, as the Bureau plans for the 2010 Census, it will be important for
it to refine its methods for enumerating people living in
unconventional housing and reporting the resulting data, in part by
properly testing and evaluating those methods. As noted earlier, the
Bureau could not properly test a key statistical technique during the
census dress rehearsal because the sample size was too small.
Moreover, while addressing the competing needs and desires of data users
will likely remain a considerable challenge, it will be important for
the Bureau to more effectively articulate its plans to avoid the
expectation gaps that occurred during 2000. The Bureau‘s plans for
collecting data on persons without conventional housing need to specify
how the Bureau plans to separately report these data.
Bureau-wide guidelines on the level of quality needed to release data
to the public, on how and when to document data limitations, and on the
circumstances under which it is acceptable to suppress data, could help
the Bureau be more accountable and consistent in its dealings with data
users and stakeholders, and help ensure that the Bureau‘s decisions
both are, and appear to be, totally objective.
Recommendations for Executive Action:
To ensure that the 2010 Census will provide public data users with more
complete, accurate, and useful information on the segment of the
population without conventional housing, we recommend that the
Secretary of Commerce direct the Director of the Bureau of the Census to
do the following.
1. Ensure that all procedures for enumerating and estimating segments of
the population without conventional housing are properly tested and
evaluated under conditions as similar to the census as possible.
2. Develop agencywide guidelines for Bureau decisions on the level of
quality needed to release data to the public, how to characterize any
limitations in the data, and when it is acceptable to suppress the data
for reasons other than protecting the confidentiality of respondents.
Ensure that these guidelines are documented, transparent, clearly
defined, and consistently applied.
3. Ensure that the Bureau‘s plans for releasing data are clearly and
consistently communicated with the public.
Agency Comments and Our Evaluation:
The Secretary of Commerce forwarded written comments from the Bureau
of the Census on a draft of this report (see app. I). The Bureau agreed
with each of our recommendations and, as indicated in the letter, is
taking steps to implement them. However, it expressed several general
concerns about our findings. The Bureau‘s principal concerns and our
response are presented below. The Bureau also suggested minor wording
changes to provide additional context and clarification. We accepted
the Bureau‘s suggestions and made changes to the text as appropriate.
The Bureau took exception to our findings concerning the adequacy of its
data quality guidelines, noting that the Bureau‘s decisions regarding
the release and characterization of emergency and transitional shelter
data were based on established guidelines for data quality. However,
the Bureau did not cite any written guidelines to support its position.
As noted in our report, Bureau officials, including the Associate
Director for Methodology and Standards, told us that the Bureau had few
written guidelines, standards, or procedures related to the quality of
data released to the public. In this report we acknowledge the Bureau‘s
tradition of high standards and procedures that yield quality data.
However, according to the Bureau, these standards are generally
undocumented and part of the agency‘s institutional knowledge. To
provide a basis for consistent decision-making and clear communication
within the Bureau and to the public, guidelines on the quality of data
released to the public must be fully documented, transparent, clearly
defined, and consistently applied.
Additionally, the Bureau said that when data do not meet an acceptable
level of quality, it considers various options for modifying its
dissemination plans. The Bureau‘s decision to delay the release of the
emergency and transitional shelter data may have been entirely
appropriate. Our concern is not that the Bureau changed its plans, but
that it could not provide us its guidelines for determining an
acceptable level of quality or clearly indicate how it determined that
the data did not meet minimal quality standards for release in SF-1.
The Bureau also commented that its decisions regarding the distribution
of data from SF-1 were well publicized and that the only change in
Bureau plans for the release of Service-Based Enumeration data was the
decision to delay release of the emergency and transitional shelter
data. This report focused on the changing plans for the release of the
emergency and transitional shelter data and noted that the Bureau never
intended to release any other data from the Service-Based Enumeration.
However, we found that the Bureau did not effectively communicate its
decisions with its partners or the public. Decisions on the release of
the emergency and transitional shelter data were contained in internal
decision memoranda. We found that these decisions were not always
reflected in new releases of the SF-1 documentation. Although Bureau
officials told us that they always intended to produce a separate
report on emergency and transitional shelter data, they did not make
this intention public when the SF-1 data were released. Some
stakeholders did not realize that the Bureau was not releasing
emergency and transitional shelter data with SF-1 until they examined
the SF-1 data. As we stated in our report, these communication problems
can undermine stakeholder and public confidence in the Bureau and its
products.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
from its issue date. At that time, we will send copies of this report
to the Chairman of the House Committee on Government Reform, the
Chairman of the Subcommittee on Civil Service, Census and Agency
Organization, the Secretary of Commerce, and the Director of the Bureau
of the Census. Copies will be made available to others on request. This
report will also be available at no charge on GAO‘s home page at
[hyperlink, http://www.gao.gov].
Please contact me on (202) 512-6806 or by e-mail at daltonp@gao.gov if
you have any questions. Other key contributors to this report were
Robert Goldenkoff, Timothy Wexler, Elizabeth Powell, Chris Miller, James
Whitcomb, Ty Mitchell, Robert Parker, and Michael Volpe.
Signed by:
Patricia A. Dalton:
Director:
Strategic Issues:
[End of section]
Appendix I: Comments from the Secretary of Commerce:
The Secretary Of Commerce:
Washington, D.C. 20230:
November 21, 2002:
Ms. Patricia A. Dalton:
Director, Strategic Issues:
General Accounting Office:
Washington, DC 20548:
Dear Ms. Dalton:
The Department of Commerce appreciates the opportunity to comment on
the General Accounting Office draft report entitled "Decennial Census:
Methods for Collecting and Reporting Data on the Homeless and Others
Without Conventional Housing Need Refinement." The Department's
comments on this report are enclosed.
Sincerely,
Signed by:
Donald L. Evans:
Enclosure:
Comments from the U.S. Department of Commerce:
U.S. Census Bureau:
U.S. General Accounting Office draft report entitled Decennial Census:
Methods for Collecting and Reporting Data on the Homeless and Others
Without Conventional Housing Need Refinement.
General Comments on the Report:
While the U.S. Census Bureau agrees with the General Accounting
Office's (GAO) recommendations in this report, we take exception to
GAO's suggestion that decisions regarding the release and
characterization of emergency and transitional shelter data were based
on anything other than our consistent commitment to clearly presenting
data that conform with established guidelines for data quality.
This sentiment, which is repeated throughout the report, is expressed
in the Highlights, when GAO writes that "a key cause of the bureau's
shifting position on reporting the Service-Based Enumeration [SBE] data
appears to be its lack of well documented, transparent, clearly
defined, and consistently applied guidelines on the minimum quality
necessary for releasing data." In fact, the Census Bureau's position on
releasing SBE figures was entirely consistent and well publicized.
Importantly, the Census Bureau never planned to release SBE data as a
separate tabulation.
The Census Bureau also is concerned that GAO does not fully present the
complexities of the SBE operation and our plans for releasing data from
that operation in the report. In the pages that follow, we have drawn
attention to inaccuracies in GAO's discussion of SBE data with the hope
of clarifying our efforts to produce and disseminate the most accurate
possible data regarding this population.
Specifications for Summary File 1, which were shared with the public,
clearly indicated that data from soup kitchens, regularly scheduled
mobile food vans, shelters for victims of domestic violence, and
targeted non-sheltered outdoor locations, would be included in the
"other non-institutional group quarters" category. The only change in
the Census Bureau's plans regarding SBE data was the decision to delay
the release of emergency and transitional shelter data rather than
including it as a separate tabulation in Summary File 1 as originally
planned. This decision to delay release of a separate tabulation of
these data was entirely consistent with the Census Bureau's commitment
to releasing data only after ensuring that they meet minimum quality
guidelines.
The Census Bureau conducted the review of the data from the SBE
operation using its established and documented standard review
techniques for reasonableness and quality. These include comparisons to
independent work and findings from experts outside the Census Bureau,
assessments of other surveys, analyses of trends, literature reviews,
and consultations with experts (both public and private). When data do
not meet an acceptable level of quality, the Census Bureau will
consider various options for modifying publication plans and determine
the most appropriate way to disseminate the data. With regard to the
emergency and transitional shelter data to be released in Summary File
1, the level of concern for the quality of the data collected warranted
a modification to the publication plans.
The Census Bureau is undertaking a review of its guidelines independent
of GAO's findings in this report.
Comments on the Text of the Report:
1. Section: Page 1, Paragraph 1 - "Although the 2000 Census counted 97
percent of the population in `conventional' housing such as a house,
apartment or mobile home, there were still many people who were counted
elsewhere, including in emergency and transitional shelters and on the
street."
Comment: As written, this sentence implies that only 97 percent of the
population residing in conventional housing was counted in Census 2000,
which is not accurate. Better would be "In Census 2000, 97 percent of
the population was counted in `conventional' housing such as a house,
apartment or mobile home, and 3 percent was counted in group quarters,
such as correctional facilities, college dormitories, nursing homes,
worker dormitories, emergency and transitional shelters, and targeted
non-sheltered outdoor locations."
2. Section: Page 1, second paragraph, second sentence - "In one
initiative, known as Service-Based Enumeration, census enumerators
attempted to count these individuals at emergency and transition
shelters, soup kitchens and other places they go for services, as well
as at what the bureau calls "targeted non-sheltered outdoor locations"
(TNSOL) for people living on the street who do not use services."
Comment: For clarification the sentence should read "In one initiative,
known as Service-Based Enumeration, census enumerators attempted to
count these individuals at emergency and transitional shelters, soup
kitchens, regularly scheduled mobile food vans, as well as what the
bureau calls `targeted non-sheltered outdoor locations' (TNSOL) for
people who are living on the street at targeted locations who do not
use services."
3. Section: Page 2, first sentence on the page - "In return, some of
these organizations expected the bureau to provide data that would help
them plan and deliver health, employment, and other services directed
toward this population."
Comment: Please add the following two sentences for the record: "In
contrast, other organizations, who helped with Census 2000, opposed the
separate release of any of the Service-Based Enumeration data. They
were concerned that data could be misused as a count of the population
experiencing homelessness and lead to flawed decision-making by
policymakers."
The Census Bureau worked closely with the National Coalition for the
Homeless and other advocacy groups in developing plans and procedures
for counting and producing tabulations for people without conventional
housing. The National Coalition for the Homeless and other advocacy
organizations urged the Census Bureau to avoid the confusion and
misinterpretations of the data produced for the 1990 Census Shelter and
Street Night Enumeration. Although the Census Bureau described the
limitations of the 1990 data and issued cautions about their usage,
stakeholders voiced concern about the meaning and the appropriate use
of these data throughout the decade.
4. Section: Page 2, Paragraph 1 - "However, after the Service-Based
Enumeration was completed, the bureau, citing quality concerns, decided
not to separately report the emergency and transitional shelter data as
originally planned."
Comment: The Census Bureau modified its tabulation plans for one Census
2000 data product, Summary File 1. This modification in the
presentation of the data was based upon the review of the data, and
that review occurred well after the Service-Based Enumeration operation
was completed. The review of the data indicated serious concerns with
the quality of the data that would require extensive limitations and
caveats to accompany these data when released. All of this was not
possible within the deadlines to release Summary File 1.
5. Section: Page 2, Paragraph 1, third sentence - "This category
included the other Service-Based Enumeration categories, data on
residential care facilities providing protective oversight, staff
dormitories for nurses and interns at military and general hospitals,
and living quarters for victims of natural disasters."
Comment: The sentence should read "At the census tract level, this
category included soup kitchens, regularly scheduled mobile food vans,
shelters for victims of domestic violence, residential care facilities
providing protective oversight, staff quarters including those for
nurses and interns at military and general hospitals, and living
quarters for victims of natural disasters. At the block level, in
addition to the categories listed above, `other non-institutional group
quarters' included group homes, religious group quarters, other
nonhousehold living situations, and workers' dormitories."
6. Section: Page 3, Paragraph 1, first sentence - "The bureau's
original plan for disseminating Service-Based Enumeration data was
outlined in an April 1999 internal memo that called for the separate
release of data on 'emergency and transitional shelters,' but did not
specify why the bureau was not separately releasing the other two
components of Service-Based Enumeration - soup kitchens and mobile food
vans, and TNSOL."
Comment: The sentence should read "... but did not specify why the
bureau was not separately releasing data on the other four locations
enumerated during the Service-Based Enumeration - soup kitchens,
regularly scheduled mobile food vans, shelters for victims of domestic
violence, and targeted non-sheltered outdoor locations."
In 1990, the Census Bureau published data showing the number of people
enumerated at selected locations where homeless people could be found.
The Census Bureau observed there was confusion and misinterpretations
of the results from the 1990 Census Shelter and Street Night
Enumeration (S-Night). Although the Census Bureau described the
limitations of these data and issued cautions about their usage,
stakeholders voiced concern about the meaning and the appropriate use
of these data throughout the decade. The kinds of issues that surfaced
following the 1990 experience were key to the decision not to publish
separate tabulations for all components of the SBE.
7. Section: Page 3, Paragraph 1, fourth sentence - "Further, despite
the bureau's warnings to the contrary, the data were often
misinterpreted as a 'homeless' count."
Comment: For clarification, please add the following sentence prior to
the sentence above: "The Census Bureau conveyed to users from the
beginning that the 1990 counts were not a count of the homeless
population."
8. Section: Page 3, Paragraph 2, first sentence - "In January 2001, the
bureau changed its earlier decision to release the data from the
emergency and transitional shelters because a procedure used to refine
the Service-Based Enumeration data proved to be unreliable."
Comment: The sentence should read ". . . emergency and transitional
shelters in Summary File 1 because a procedure...."
9. Section: Page 3, Paragraph 2, third sentence - "However, because
problems with the procedure surfaced in the 2000 Census, the bureau
decided to combine the emergency and transitional shelter data with the
`other non-institutional group quarters' category that also includes
data on people enumerated in the other categories of the Service-Based
Enumeration and in several other group locations such as facilities for
victims of natural disasters."
Comment: The problems with the Service-Based Enumeration data were
identified during the review of data from that operation. This review
identified an unacceptably high nonresponse rate and level of response
bias for the service usage question asked of people enumerated at the
service locations. The Census Bureau decided it would be inappropriate
to release these data using the multiplicity estimation results to
distribute people to local areas and service facilities because they
were not statistically defensible.
10. Section: Page 4, Paragraph 1- "In the fall of 2001, the bureau
produced a special report on the emergency and transitional shelter
data-the same data that the bureau earlier stated it could not release
because of quality concerns. However, the bureau added a lengthy
discussion of its limitations and emphasized that the data should not
be interpreted as a count of the homeless population."
Comment: This representation is incorrect. Based on the continued
review of the emergency and transitional shelter data, and direction
from the Census Bureau's Disclosure Review Board, data in the October
report differed from the April 1999 dissemination plan for SF-1. These
differences include such changes as follows:
(1) Two age groups (under 18 years and 18 years and over) instead of
three age groups (under 18 years, 18 to 64 years, and 65 years and
over).
(2) Tables for metropolitan areas and places of 100,000 or more
population will show only those areas with 100 or more people in
emergency and transitional shelters for two broad age groups.
(3) Data for the population in emergency and transitional shelters by
age, race and Hispanic origin, and sex will only be shown for the
United States as a whole.
11. Section: Page 8, Paragraph 1, last sentence - "This category also
included residential care facilities providing protective oversight,
staff dormitories for nurses and interns at military and general
hospitals, and living quarters for victims of natural disasters."
Comment: The sentence should read ". . . residential care facilities
providing protective oversight, shelters against domestic violence,
staff quarters including those for nurses...."
12. Section: Page 8, Footnote 4 - "The SF-1 is the summary file in
which the bureau presents population and housing data for the total
population. SF-1 data are the first product released after the census."
Comment: The first data product released from Census 2000 was the
Census 2000 Redistricting Data Summary File.
13. Section: Page 9, Paragraph 1 - "The bureau's data dissemination
plans took into account the recommendations of the Commerce Secretary's
2000 Census Advisory Committee, a panel that included representatives
of advocacy and other groups that met periodically to review the
bureau's plans."
Comment: The Census Bureau's advisory committees include
representatives from advocacy groups as well as representatives from
organizations that represent local governments, such as the National
Association of Counties, National Association of Towns and Townships,
National League of Cities, U.S. Conference of Mayors, and the
International City/County Management Association.
14. Section: Page 9, Paragraph 1, last sentence - "In its January 1999
final report, the Committee recommended that special attention be paid
to tabulating the results of the Service-Based Enumeration and targeted
outdoor enumerations so that they could not be aggregated and used as a
homeless count."
Comment: Please add "Census 2000 Advisory" before committee.
15. Section: Page 10, Figure 1.
Comment: "Shelters for victims of domestic violence" should be included
in the "other non-institutional group quarters" category. For
clarification, prisons should be changed to correctional institutions,
ship crews should be changed to crews of maritime vessels, and mobile
food vans should be changed to regularly scheduled mobile food vans.
16. Section: Page 10, last sentence on page - "Bureau officials told us
that the decision to exclude a separate emergency and transitional
shelter count was made between December 2000 and January 2001, by the
Director of Decennial Census with input from Associate Director
Decennial Census, the Population Division, and the Decennial
Statistical Studies Division."
Comment: The sentence should read - "Bureau officials told us that the
decision to exclude a separate emergency and transitional shelter count
in Summary File 1...."
Add Associate Director for Demographic Programs and the Decennial
Management Division.
17. Section: Page 11, Paragraph 1, third sentence - "The multiplicity
estimation procedure was based on information from those who were
counted on the number of times they used the service facilities in the
prior week."
Comment: The sentence should read ". . . from those who were counted
and on the number...."
18. Section: Page 12, Paragraph 1, last sentence - "Service-Based
Enumeration data were the only data that were to be released in SF-1
under separate reporting categories that the bureau decided to combine
with another category."
Comment: Emergency and transitional shelter data, not service-based
enumeration data, were the only data that were originally to be
separately identified in SF 1, but were collapsed into another
category. Data for other locations enumerated in the service-based
enumeration were always collapsed into the category called "other non-
institutional group quarters" in Summary File 1.
This change was based upon the Census Bureau's increasing concerns that
the census results from people enumerated at emergency and transitional
shelters, without the appropriate qualifiers and other limitations,
would be misinterpreted. During this time, it was impossible to include
the limitations and qualifiers in the technical documentation of
Summary File 1. As a result, the Census Bureau decided to issue a
special report later on the results of the enumeration of emergency and
transitional shelters with the appropriate caveats.
19. Section: Page 12, Paragraph 2, fourth sentence - "This report
separately identified emergency and transitional shelter data at the
census tract level (small statistical subdivision of counties), but not
targeted nonsheltered outdoor locations or soup kitchens and mobile
food vans."
Comment: This report separately identified emergency and transitional
shelter data for various levels of geography down to the census tract
level (small statistical subdivisions of counties) with 100 or more
people in emergency and transitional shelters. The report did not
include data for the population in targeted non-sheltered outdoor
locations, soup kitchens, regularly scheduled mobile food vans, and
shelters for domestic violence.
20. Section: Page 13, last sentence at top of page - "Moreover, the
emergency and transitional shelter data at the census tract level are
not in the hard copy, but rather in the Internet version of the
report."
Comment: All Census 2000 data at the tract level are available on the
Internet and are not available in printed reports.
21. Section: Page 13, Paragraph 1, first sentence - "The October report
contains the data that were to be released under the April 1999
dissemination plan for SF-1"
Comment: See comment in response to Item 10 above.
22. Section: Page 13, Paragraph 2, first sentences - "The October 2001
report also identified reasons the bureau did not (and never planned
to) separately release data on people counted at targeted non sheltered
outdoor locations, soup kitchens, and regularly scheduled mobile food
vans":
Comment: The sentence should read ". . . soup kitchens, regularly
scheduled mobile food vans, and shelters for victims of domestic
violence":
23. Section: Page 17, Paragraph 1, last sentence - "Some partners
stated that they did not know that the bureau never intended to report
the targeted non-sheltered outdoor location data."
Comment: The Census Bureau worked with representatives of the Census
2000 Advisory Committee and the Association of Public Data Users (APDU)
in developing and finalizing the specifications for Summary File 1. All
draft specifications released for Summary File 1 showed the following
service-based enumeration locations collapsed into the "other non-
institutional group quarters" category: soup kitchens, regularly
scheduled mobile food vans, shelters for victims of domestic violence,
and targeted non-sheltered outdoor locations.
Responses to GAO Recommendations:
Recommendation 1: "Ensure that all procedures for enumerating and
estimating segments of the population without conventional housing are
properly tested and evaluated under conditions as similar to the census
as possible."
Census Bureau Response: The Census Bureau concurs with the
recommendation. This work is underway as part of the research and
testing program for the 2010 Census.
Recommendation 2: "Develop agency-wide guidelines for bureau decisions
on the level of quality needed to release data to the public, how to
characterize any limitations in the data, and when it is acceptable to
suppress the data for reasons other than protecting the confidentiality
of respondents. Ensure that these guidelines are documented,
transparent, clearly defined, and consistently applied."
Census Bureau Response: The Census Bureau concurs with the
recommendation. In order to continue to maintain the long tradition of
producing high-quality data, the Census Bureau has asked the
Methodology and Standards Council to recommend statistical and survey
quality guidelines for surveys and censuses.
Recommendation 3: "Ensure that the bureau's plans for releasing data
are clearly and consistently communicated with the public."
Census Bureau Response: The Census Bureau concurs with the
recommendation. The Census Bureau will continue to work with data users
who have different requirements and positions on the release of the
data in order to develop tabulation plans that meet the needs of
stakeholders and users of the data. As discussed in this report, the
release of data from the service-based operation generated two very
different positions from data users, ranging from some users wanting
data released in a similar manner as the 1990 census to other users
opposing the release of any data from the service-based enumeration.
The Census Bureau will continue to work with stakeholders to try to
reconcile these differences and then develop publication plans based
upon these needs.
[End of section]
Related GAO Products on the Results of the 2000 Census and Lessons
Learned for a More Effective Census in 2010:
2000 Census: Refinements to Full Count Review Program Could Improve
Future Data Quality. GAO-02-562. Washington, D.C.: July 3, 2002.
2000 Census: Coverage Evaluation Matching Implemented As Planned,
but Census Bureau Should Evaluate Lessons Learned. GAO-02-297.
Washington, D.C.: March 14, 2002.
2000 Census: Best Practices and Lessons Learned for a More Cost-
Effective Nonresponse Follow-Up. GAO-02-196. Washington, D.C.:
February 11, 2002.
2000 Census: Coverage Evaluation Interviewing Overcame Challenges,
but Further Research Needed. GAO-02-26. Washington, D.C.: December 31,
2001.
2000 Census: Analysis of Fiscal Year 2000 Budget and Internal Control
Weaknesses at the U.S. Census Bureau. GAO-02-30. Washington, D.C.:
December 28, 2001.
2000 Census: Significant Increase in Cost Per Housing Unit Compared
to 1990 Census. GAO-02-31. Washington, D.C.: December 11, 2001.
2000 Census: Better Productivity Data Needed for Future Planning and
Budgeting. GAO-02-4. Washington, D.C.: October 4, 2001.
2000 Census: Review of Partnership Program Highlights Best Practices
for Future Operations. GAO-01-579. Washington, D.C.: August 20, 2001.
Decennial Censuses: Historical Data on Enumerator Productivity Are
Limited. GAO-01-208R. Washington, D.C.: January 5, 2001.
2000 Census: Information on Short- and Long-Form Response Rates.
GAO/GGD-00-127R. Washington, D.C.: June 7, 2000.
[End of section]
Footnotes:
[1] U.S. General Accounting Office, Decennial Census: Methods for
Collecting and Reporting Hispanic Subgroup Data Need Refinement, GAO-03-
228 (Washington, D.C.: Jan. 17, 2003).
[2] For further information see, U.S. General Accounting Office, 1990
Census: Limitations in Methods and Procedures to Include the Homeless,
GAO/GGD-92-1 (Washington, D.C.: Dec. 30, 1991).
[3] For information on the conduct of Service-Based Enumeration see,
U.S. General Accounting Office, 2000 Census: Progress Report on the
Mail Response Rate and Key Operations, GAO/T-GGD/AIMD-00-136
(Washington, D.C.: Apr. 5, 2000).
[4 For additional information see, U.S. General Accounting Office, 2000
Census: Review of Partnership Program Highlights Best Practices for
Future Operations, GAO-01-579 (Washington, D.C.: Aug. 20, 2001).
[5] The SF-1 is the summary file in which the Bureau presents
population and housing data for the total population. Other than the
Census 2000 Redistricting Data Summary File, the SF-1 file is the first
product released after the census.
[6] For more information on the 1990 count see, U.S. General Accounting
Office, Counting the Homeless: Limitations of 1990 Census Results and
Methodology, GAO/T-GGD-91-29 (Washington, D.C.: May 9, 1991) and 1990
Census: Limitations in Methods and Procedures to Include the Homeless,
GAO/GGD-92-1 (Washington, D.C.: Dec. 30, 1991).
[7] One example was the National Law Center On Homelessness and
Poverty, et. al., v. Brown, where plaintiffs alleged that the
Department of Commerce effectively excluded the nation‘s homeless
population from the 1990 decennial census in violation of the
Constitution and the Administrative Procedure Act. The District Court
denied the plaintiff‘s claim (1994 WL 521334) (D.D.C. Sept. 15, 1994)
and the Court of Appeals dismissed the lawsuit for lack of standing. 91
F.3d 178 (D.C. Cir. 1996).
[8] Although the Bureau changed its plan in January 2001, the technical
documentation for SF-1, released at the same time, still indicated that
emergency and transitional shelter data would be separately reported.
[9] The dress rehearsal for the 2000 Census was conducted in
Sacramento, California, City of Columbia, South Carolina, and Menominee
County, Wisconsin, including the Menominee Indian Reservation. The
dress rehearsal was designed to test the overall design of the 2000
Census.
[10] U.S. Bureau of the Census, Population in Emergency and
Transitional Shelters (PHC-T-12) (Washington, D.C.: Oct. 30, 2001).
[hyperlink, http://www.census.gov/population/www/cen2000/phc-t12.html]
[11] U.S. General Accounting Office, Internal Control Standards,
Internal Control Management and Evaluation Tool (August 2001) and U.S.
General Accounting Office, Internal Control Standards for Internal
Control in the Federal Government (November 1999).
[12] U.S. Department of Commerce, Technical Paper 32, Standards for
Discussion and Presentation of Errors in Data (March 1974), p. 3.
[13] Issued at 67 Fed. Reg. 8452 (Feb. 22, 2002).
[End of section]
GAO‘s Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO‘s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO‘s Web site [hyperlink,
http://www.gao.gov] contains abstracts and full text files of current
reports and testimony and an expanding archive of older products. The
Web site features a search engine to help you locate documents using
key words and phrases. You can print these documents in their entirety,
including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ’Today‘s Reports,“ on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
[hyperlink, http://www.gao.gov] and select ’Subscribe to daily E-mail
alert for newly released products“ under the GAO Reports Order GAO
Products heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537
Fax: (202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov:
(202) 512-4800:
U.S. General Accounting Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
