Data Quality
Expanded Use of Key Dissemination Practices Would Further Safeguard the Integrity of Federal Statistical Data Gao ID: GAO-06-607 May 31, 2006In 2003, the Bureau of the Census (Bureau) changed the day and location of the release of its Income and Poverty Estimates. Some data users believed the change was an effort to suppress unfavorable news and questioned the Bureau's data dissemination practices. GAO was asked to assess whether (1) the Bureau adhered to its dissemination practices for the 2003 and later releases, and (2) the Bureau and 13 other federal statistical agencies follow data release practices recommended by the National Research Council (NRC). GAO reviewed the Bureau's dissemination process for the 2003 thru 2005 Income and Poverty Estimates.
While not all of the Bureau's data dissemination practices are documented, GAO was able to determine through discussions with Bureau officials and review of available documentation, that the Bureau adhered to most of its long-standing data release practices. However, the Bureau did depart from the traditional day and location for the release of the Income and Poverty Estimates report in 2003 and subsequent years. According to the Bureau, the day of the 2003 release was changed because of a delay in producing a companion report, and the location was changed from Washington, D.C., to Suitland, Maryland, in part, because the Director of the Census Bureau stated that he wanted to raise awareness that the construction of its new headquarters had just started. Some of the Bureau's documented practices, such as guidance on who has authority to choose the release date and location, lacked specificity. Also, the Bureau's documented Income and Poverty practices are outdated as they are contained in a 21-year-old memo. The Bureau is updating it, to among other things, reflect current technology. Most of the 14 statistical agencies in GAO's review generally adhered to NRC's guidance, important for (1) the wide dissemination of data, and (2) maintaining a strong position of independence. Still, there were some notable gaps. For example, 6 of the 14 agencies lacked dissemination policies (as recommended by NRC) that promote the regular and frequent release of major findings from an agency's statistical program. The Office of Management and Budget (OMB), in concert with other statistical agencies, is developing governmentwide guidance on the release and dissemination of statistical products that, according to OMB officials, parallels NRC's and other generally accepted release practices. OMB's guidance could foster more consistent adherence to practices that promote broader dissemination of statistical data and enhance its credibility, especially to the extent they address gaps GAO found between agencies' data dissemination practices and NRC's guidance.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-607, Data Quality: Expanded Use of Key Dissemination Practices Would Further Safeguard the Integrity of Federal Statistical Data
This is the accessible text file for GAO report number GAO-06-607
entitled 'Data Quality: Expanded Use of Key Dissemination Practices
Would Further Safeguard the Integrity of Federal Statistical Data'
which was released on June 5, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
May 2006:
Data Quality:
Expanded Use of Key Dissemination Practices Would Further Safeguard the
Integrity of Federal Statistical Data:
Data Quality:
GAO-06-607:
GAO Highlights:
Highlights of GAO-06-607, a report to congressional requesters.
Why GAO Did This Study:
In 2003, the Bureau of the Census (Bureau) changed the day and location
of the release of its Income and Poverty Estimates. Some data users
believed the change was an effort to suppress unfavorable news and
questioned the Bureau‘s data dissemination practices. GAO was asked to
assess whether (1) the Bureau adhered to its dissemination practices
for the 2003 and later releases, and (2) the Bureau and 13 other
federal statistical agencies follow data release practices recommended
by the National Research Council (NRC). GAO reviewed the Bureau‘s
dissemination process for the 2003 thru 2005 Income and Poverty
Estimates.
What GAO Found:
While not all of the Bureau‘s data dissemination practices are
documented, GAO was able to determine through discussions with Bureau
officials and review of available documentation, that the Bureau
adhered to most of its long-standing data release practices. However,
the Bureau did depart from the traditional day and location for the
release of the Income and Poverty Estimates report in 2003 and
subsequent years. According to the Bureau, the day of the 2003 release
was changed because of a delay in producing a companion report, and the
location was changed from Washington, D.C., to Suitland, Maryland, in
part, because the Director of the Census Bureau stated that he wanted
to raise awareness that the construction of its new headquarters had
just started. Some of the Bureau‘s documented practices, such as
guidance on who has authority to choose the release date and location,
lacked specificity. Also, the Bureau‘s documented Income and Poverty
practices are outdated as they are contained in a 21-year-old memo. The
Bureau is updating it, to among other things, reflect current
technology.
Figure: Bureau Press Conference for Release of Income and Poverty
Estimates:
[See PDF for Image]
[End of Figure]
Most of the 14 statistical agencies in GAO‘s review generally adhered
to NRC‘s guidance, important for (1) the wide dissemination of data,
and (2) maintaining a strong position of independence. Still, there
were some notable gaps. For example, 6 of the 14 agencies lacked
dissemination policies (as recommended by NRC) that promote the regular
and frequent release of major findings from an agency‘s statistical
program. The Office of Management and Budget (OMB), in concert with
other statistical agencies, is developing governmentwide guidance on
the release and dissemination of statistical products that, according
to OMB officials, parallels NRC‘s and other generally accepted release
practices. OMB‘s guidance could foster more consistent adherence to
practices that promote broader dissemination of statistical data and
enhance its credibility, especially to the extent they address gaps GAO
found between agencies‘ data dissemination practices and NRC‘s
guidance.
What GAO Recommends:
The Secretary of Commerce should direct the Bureau to fully document
its dissemination practices for the Income and Poverty Estimates. GAO
also recommends that OMB consider (1) how to address gaps identified
between agencies‘ dissemination practices and NRC‘s guidance, and (2)
how OMB‘s proposed guidance should address documentation, coverage, and
other questions noted in this report. In its comments on a draft of
this report, Commerce neither agreed nor disagreed with GAO‘s
recommendation but reiterated GAO‘s finding that the Bureau is updating
its practices for releasing the Income and Poverty Estimates. OMB did
not have comments.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-607].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Brenda S. Farrell at
(202) 512-6806 or farrellb@gao.gov.
[End of Section]
Contents:
Letter:
Results in Brief:
Background:
Release of Income and Poverty Estimates Adhered to Most of the Bureau's
Data Dissemination Practices:
Governmentwide Guidance Is Being Developed That May Improve Statistical
Agencies' Data Dissemination Practices:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Commerce:
Appendix III: GAO Contact and Acknowledgments:
Related GAO Reports:
Tables:
Table 1: Income and Poverty Estimates Have Traditionally Been Released
on a Tuesday or a Thursday at the National Press Club:
Table 2: Most Statistical Agencies Reported That They Generally Adhered
to the National Research Council's Guidance for Releasing Information:
United States Government Accountability Office:
Washington, DC 20548:
May 31, 2006:
The Honorable Wm. Lacy Clay:
Ranking Minority Member:
Subcommittee on Federalism and the Census:
Committee on Government Reform:
House of Representatives:
The Honorable Carolyn B. Maloney:
House of Representatives:
Data collected and disseminated by federal statistical agencies are a
critical strategic asset because they provide indicators of the
economic and social well-being of the nation and help inform agencies'
rule-making activities. Given the widespread use and the impact of
federal information, the data itself, the timing of reports, and any
news releases that accompany them must be accurate and objective.
Moreover, statistical agencies must be viewed as credible to secure the
public's trust. One way statistical agencies foster credibility is by
maintaining a position of independence; that is, taking appropriate
steps that help ensure their data products and the timing of their
release are free from even an appearance of political influence.
Because of the sensitivity of certain statistical information,
seemingly mundane actions, such as the timing of a data release, can be
called into question, as interested parties may perceive the change as
a maneuver to obscure or manipulate the information for partisan
purposes. The Department of Commerce, which has a number of data
gathering programs under its purview, oversees the U.S. Bureau of the
Census (Bureau). Among its many surveys, the Bureau has responsibility
to report estimates of the nation's income and poverty. In 2003, the
Bureau changed the day and location for releasing this information.
Because the estimates showed that poverty rates had risen, some data
users believed that the change was an effort to suppress unfavorable
news, and called into question the Bureau's practices for disseminating
such data, and the Department of Commerce's role in the 2003 release.
This letter responds to your request that we review the Bureau's level
of independence in releasing its annual Income and Poverty Estimates
over the last several years. As agreed with your offices, we determined
the extent to which (1) the Bureau adhered to its dissemination
practices for the issuance of the 2003 annual Income and Poverty
Estimates and subsequent releases in 2004 and 2005, and (2) the Bureau
and 13 other federal statistical agencies follow data dissemination
practices that the National Academy of Sciences' National Research
Council (NRC) recommend in a 2005 report.[Footnote 1]
To address the first objective, we reviewed relevant program documents
and interviewed Bureau officials responsible for disseminating the
Income and Poverty Estimates and other data releases. The dissemination
process includes the steps from approval of the report content up to
and including public distribution of the report. For the second
objective, we surveyed key officials at the Bureau and 13 other federal
statistical agencies collecting information on their practices for
releasing data. We compared the 14 agencies' practices to those
developed by NRC that are important for (1) the wide dissemination of
data, and (2) maintaining a strong position of independence. According
to NRC, elements within these practices facilitate the provision of
timely, credible, and politically neutral information into the hands of
data users. Appendix I provides additional details on our scope and
methodology. We conducted our work between March 2005 and April 2006 in
accordance with generally accepted government auditing standards.
Results in Brief:
While not all of the Bureau's data dissemination practices are
documented, we were able to determine through discussions with Bureau
officials and review of available documentation, that the Bureau
adhered to most of its long-standing data release practices. In
changing the date and location of the 2003 and subsequent releases of
the Income and Poverty Estimates, the Bureau did depart from its
tradition of releasing this information on a Tuesday or Thursday at a
news conference at the National Press Club. That said, under the
Bureau's documented data dissemination practices (1) there is no
requirement for the Bureau to release this information at a particular
location on a given day, and (2) no particular official is designated
authority to choose the release date and location. Also, the Bureau's
documented data dissemination practices for the Income and Poverty
Estimates are contained in a memo from 1985. The Bureau is updating
them, to among other things, reflect current technology.
Bureau officials stated that the date of the 2003 release was changed
from September 23rd to September 26th for several reasons, including
delays in producing a companion report on supplemental measures of
expenditures, consumption, and poverty. According to Bureau officials
we interviewed and available documents, the Commerce Department's Under
Secretary for Economic Affairs wanted to release the two reports
simultaneously to broaden the public's understanding of social well-
being, a decision that was consistent with the Bureau's ongoing effort
to provide alternative measures of poverty.
In 2004, the Bureau moved up the release of the Income and Poverty
Estimates to August of that year so it would coincide with the release
of data from the American Community Survey.[Footnote 2] According to
Bureau officials, this was done in an attempt to head off the confusing
press coverage that occurred the previous year when estimates from
Income and Poverty Estimates did not always match estimates from the
American Community Survey. In 2004, when the Bureau issued the Income
and Poverty Estimates and American Community Survey numbers at the same
time, the Bureau's news release explained the methodological and other
factors that could account for any differences.
Bureau officials stated that in 2003, the location of the press
conference for the Income and Poverty Estimates was changed from
Washington, D.C., to the Bureau's Suitland, Maryland campus, in part,
because the Director of the Census Bureau wanted to raise awareness
that the construction of its new headquarters building had just
started. According to Bureau officials, future releases of the annual
Income and Poverty Estimates are to occur at the Suitland, Maryland
campus in late August.
Most of the 14 statistical agencies we included in our review reported
general adherence to NRC's guidance, important for (1) the wide
dissemination of data, and (2) maintaining a strong position of
independence. Still, there were some noteworthy gaps. On the one hand,
for example, all 14 agencies (in accordance with NRC guidance) had
multiple avenues for disseminating data, released data in a variety of
formats, and had policies to guide what data should be preserved and
how they should be archived. On the other hand, adherence to NRC's
practices was not universal. For example, 6 of the 14 agencies lacked
dissemination practices that promote the regular and frequent release
of major findings from an agency's statistical programs to the public
via the media, the Internet, and other means.
The Office of Management and Budget (OMB), in concert with the federal
statistical agencies, is developing a governmentwide directive on the
release and dissemination of statistical products that, according to
OMB officials, parallels NRC's and other generally accepted
dissemination practices. OMB officials indicated the guidance is
intended to help ensure statistical products are policy-neutral,
timely, and accurate. To the extent that statistical agencies
appropriately follow these practices, the directive could promote more
consistent adherence to practices that facilitate broader dissemination
of statistical data and enhance its credibility. For example, OMB's
directive could help replace the patchwork of agency-specific guidance
with a more transparent, commonly accepted, and consistently applied
framework for disseminating data. OMB plans to release the directive
for public comment in the spring of 2006.
We are recommending that the Secretary of Commerce direct the Bureau
to, as part of its efforts to update its practices for releasing the
Income and Poverty Estimates, fully document those practices. We are
also making recommendations to the Director of OMB, as OMB completes
work on its draft dissemination directive, to consider how best to
address the gaps we identified between agencies' data dissemination
practices and NRC's guidance, as well as certain questions concerning
coverage, documentation, flexibility, monitoring, and the posting of
data.
The Secretary of Commerce provided written comments on a draft of this
report (see app. II). While Commerce neither agreed nor disagreed with
our recommendation for the Bureau to fully document its key data
dissemination practices for releasing the Income and Poverty Estimates,
Commerce reiterated the point we made in our report that the Bureau is
updating its practices for releasing the Income and Poverty Estimates.
Commerce noted that the updated document--which details the
dissemination practices for the Income and Poverty Estimates--is under
review. The Bureau plans to issue it prior to the next release of the
Income and Poverty Estimates expected in August 2006.
The Director of OMB did not have comments on the recommendations to
them. However, OMB officials provided suggestions for technical
corrections and we revised the report to reflect these suggestions as
appropriate.
Background:
In September 2003, the Bureau broke from its tradition of releasing its
Income and Poverty Estimates on a Tuesday or Thursday at a news
conference at the National Press Club (see table 1). The data were
instead released at a news conference on a Friday at Bureau
Headquarters in Suitland, Maryland. Although the Bureau provided the
media and other attendees with bus service from the National Press Club
to Suitland, because the data showed that poverty levels had risen,
some data users expressed concern that the change in day and location
was an attempt to suppress unfavorable information by releasing it at a
more remote location and before a weekend, when the public tends to pay
less attention to the news.
Table 1: Income and Poverty Estimates Have Traditionally Been Released
on a Tuesday or a Thursday at the National Press Club:
Year: 1996;
Date/Day: September 26 - Thursday;
Location: National Press Club.
Year: 1997;
Date/Day: September 29 - Monday;
Location: National Press Club.
Year: 1998;
Date/Day: September 24 - Thursday;
Location: National Press Club.
Year: 1999;
Date/Day: September 30 - Thursday;
Location: National Press Club.
Year: 2000;
Date/Day: September 26 - Tuesday;
Location: National Press Club.
Year: 2001;
Date/Day: September 25 - Tuesday;
Location: National Press Club.
Year: 2002;
Date/Day: September 24 - Tuesday;
Location: National Press Club.
Year: 2003;
Date/Day: September 26 - Friday;
Location: Census Bureau.
Year: 2004;
Date/Day: August 26 - Thursday;
Location: Census Bureau.
Year: 2005;
Date/Day: August 30 - Tuesday;
Location: Census Bureau.
Source: U.S. Census Bureau.
[End of table]
The Income and Poverty Estimates, like other kinds of federal
statistical information, provide key measures of the health and well-
being of our society. As a result, the data need to be accurate,
timely, accessible, relevant, and objective. At the same time,
according to NRC, the manner in which agencies release the data is also
important, and needs to be free from even an appearance of bias and
political manipulation. Failure to meet this goal could undermine
public confidence in the information and erode an agency's credibility.
That said, although various guidance and laws have been developed to
safeguard the overall quality of federal data, few governmentwide
provisions directly address the data dissemination process itself, and
agencies have largely been left to develop their own practices. For
example, while OMB's Statistical Policy Directive Number 3,
"Compilation, Release, and Evaluation of Principal Federal Economic
Indicators" provides detailed guidance on the dissemination of data, it
only applies to 38 market sensitive principal economic indicators.
Statistical Policy Directive Number 3 is highly regarded in the
statistical community because it provides statistical agencies with
comprehensive data dissemination guidance, requiring agencies to, among
other actions, promptly release data according to an established
schedule, and announce and fully explain any schedule changes in
advance.
Under the Information Quality Act,[Footnote 3] OMB was required to
issue governmentwide guidelines that provide policy and procedural
guidance to federal agencies for ensuring and maximizing the quality,
objectivity, utility, and integrity of information disseminated by
federal agencies. OMB's guidelines, issued in final form in February
2002, directed agencies covered by the act (statistical agencies and
most others) to issue their own quality guidelines. OMB's guidelines
imposed certain core responsibilities on agencies, including
incorporating quality into their information dissemination practices.
OMB noted that quality consists of several dimensions, including
objectivity (which focuses on whether the disseminated information is
accurate, reliable, and unbiased in presentation and substance).
More generally, OMB helps ensure that the activities of the statistical
agencies are in line with federal statistical policy by coordinating
agency budget requests and interagency groups working on statistical
issues, issuing statistical standards, and reviewing agency requests to
collect information.
This report is the latest of several studies we have issued on the
quality of federal data. See Related GAO Reports at the end of this
report for a list of selected products we have issued to date.
Release of Income and Poverty Estimates Adhered to Most of the Bureau's
Data Dissemination Practices:
While not all of the Bureau's data dissemination practices are
documented, we were able to determine through discussions with Bureau
officials and review of available documentation, that the Bureau
adhered to most of its long-standing data release practices. In
changing the date and location of the 2003 and subsequent releases of
the Income and Poverty Estimates, the Bureau did depart from its
tradition of releasing this information on a Tuesday or Thursday at a
news conference at the National Press Club. That said, under the
Bureau's documented data dissemination practices (1) there is no
requirement for the Bureau to release this information at a particular
location on a given day and, (2) no particular official is designated
authority to choose the release date and location. Bureau officials
stated that the date of the 2003 release was changed from September
23rd to September 26th for several reasons, including delays in
producing a companion report on supplemental measures of expenditures,
consumption, and poverty that was to be released at the same time.
Also, the 2004 and 2005 estimates were released a month earlier than in
prior years to coincide with the release of data from the American
Community Survey. The documented practices for disseminating the Income
and Poverty Estimates are contained in a memo that is 21 years old so
the Bureau is updating them, to among other things, reflect current
technology.
The Bureau's Income and Poverty Estimate Dissemination Practices Are
Derived from Several Sources:
The Bureau has several sources of documented, agencywide practices for
disseminating data to the public. For example, in accordance with OMB's
guidelines for implementing the Information Quality Act, the Bureau
developed its own set of quality guidelines that include provisions
aimed at ensuring the objectivity and integrity of its data.[Footnote
4] The Bureau also has a series of data dissemination practices
available on its Intranet site and it has issued four standards
governing the dissemination of data products,[Footnote 5] including
Dissemination of Census and Survey Data Products.[Footnote 6] We found
that the only documented practices specific to the release of the
Income and Poverty Estimates are contained in a 1985 memorandum that
was included as one of several appendixes to the Bureau's
Administrative Manual. The manual provides Bureau policy on the release
of data and guidance for divisions to follow in responding to requests
for such information.
The 1985 memorandum, which was signed by the Director of the Census
Bureau at the time, includes eight broad steps, covering the process
for disseminating the Income and Poverty Estimates. The eight steps
include the time period from approval of the report content up to and
including distributing the report at the press conference.
1. The Associate Director for Demographic Fields approves the final
content of the report prepared by the Population Division after review
by the Statistical Methods Division.
2. The Public Information Office receives a copy of the final content
to draft a press release. This draft release is approved within the
Census Bureau, by the Public Affairs Specialist in the Under Secretary
for Economic Affairs' office, and by the Commerce Department's
newsroom.
3. The report is prepared for camera-ready form.
4. Camera-ready copy is sent to the printer.
5. When the completion time for this report is known, the Census Bureau
establishes the release date and time with Commerce Department
concurrence.
6. Approximately 48 hours before report release date and time, the
Census Bureau briefs the Deputy Secretary for Economic Affairs on the
principal findings.
7. The Census Bureau makes the report and accompanying press release
available to the media on the established date at 9 a.m. for 10 a.m.
release.[Footnote 7]
8. The Census Bureau distributes the report and press release to the
Congress and the OMB at the same time as the media.
Release of 2003 and Subsequent Income and Poverty Estimates:
In releasing the 2003 Income and Poverty Estimates, the Bureau adhered
to most of its data dissemination practices. The change in the date and
location of the 2003 and subsequent releases of the Income and Poverty
Estimates was a departure from the Bureau's tradition of releasing this
information on a Tuesday or Thursday at a news conference at the
National Press Club. That said, under the Bureau's documented data
dissemination practices there is no requirement for the Bureau to
release this information at a particular location on a given day. Based
on our review of available documentation and our interviews with
officials involved with the Income and Poverty Estimates, the Bureau
followed the steps in the 1985 memo in the 2003, 2004 and 2005
releases, with the exception of the release time as previously
described.
While the Bureau complied with its documented practices for the
dissemination of the Income and Poverty Estimates, they lacked
specificity. For example, clear and specific documentation does not
exist for how and when the release date and location are to be
determined for the Income and Poverty Estimates and who should make
those decisions. In actuality, as discussed in greater detail
subsequently, in 2003, the Director of the Census Bureau chose the
location and the Associate Director for Communications chose the date.
However, because this was not thoroughly documented (the 1985 memo only
provides general guidance), it is unclear to the public who made these
decisions and how they were made.
Furthermore, Bureau officials told us that they did not retain any
internal memos or e-mails that documented the decision to change the
2003 Income and Poverty Estimates release, which would have provided
evidence to support the Bureau's narrative of the events leading up to
the release.
According to the Bureau, Delays in the Companion Report Caused Change
in Timing of the 2003 Release:
Based on our review of available Bureau documents and interviews with
key Bureau officials, several factors led to the change in the timing
of the release of the 2003 and subsequent Income and Poverty Estimates.
The Chief of the Census Bureau's Housing and Household Economic
Statistics Division[Footnote 8] at the time of the 2003 release of
Income and Poverty Estimates, and other senior officials we spoke to,
stated that the 2003 Income and Poverty Estimates release was different
from years past because the Bureau decided earlier that year to issue
the report at the same time as a multi-agency report on supplemental
measures of expenditures, consumption, and poverty.[Footnote 9] This
decision was made before the findings of the Income and Poverty
Estimates report were known. Bureau officials stated that although the
original target date for releasing both reports was September 23, 2003,
complications with finalizing the supplemental measures report kept it
from being ready for release on that day.
According to Bureau officials and documents we reviewed, because the
supplemental measures of expenditures, consumption, and poverty report
involved several statistical agencies, there was a different clearance
process than that used for the Income and Poverty Estimates report. As
a result, while the Bureau had completed its review of the latter
report, all the members of a steering committee still needed to review
the report on supplemental measures.
At the same time, based on our discussions with Bureau officials
involved with the Income and Poverty Estimates report, as well as
available documents, the Commerce Department's Under Secretary for
Economic Affairs wanted to release both reports simultaneously in an
effort to broaden the public's understanding of social well-being. The
Under Secretary's decision was consistent with the Bureau's ongoing
effort to provide alternative estimates of poverty, which itself
stemmed from a 1995 report by the National Academy of Sciences that
recommended revising how poverty is measured.
Because of the additional time required to clear the supplemental
measures report, Bureau officials responsible for the Income and
Poverty Estimates asked for a later date to issue their report.
Consequently, the Bureau's Associate Director for Communications, with
the Director's consent, scheduled Friday, September 26, 2003, as the
release date for the Income and Poverty Estimates, and both reports
were issued on that date.[Footnote 10]
Under the Bureau's guidance for dealing with the media, Census Bureau
analysts are to arrange their work schedules to be available for
inquiries for 2 to 3 days after a data release. This is why, prior to
2003, the Bureau tended to release the Income and Poverty Estimates
earlier in the week: it obviated the need for analysts to work on the
weekend. Additionally, Bureau officials said that because of the
Internet and cable television, the news cycle is no longer viewed as a
cycle and has instead become a 24 hours a day, 7 days a week operation.
Thus, many of the media's inquiries occur the same day as the data are
released. While it seldom does so, the Bureau has released other
reports on Fridays, such as its 2001 health insurance report.[Footnote
11]
For the 2004 and 2005 releases of Income and Poverty Estimates, the
data were released in August at the same time as data from the American
Community Survey. Bureau officials reported the Income and Poverty
Estimates (which come from the Bureau's Current Population Survey) are:
one of several sources of income and poverty information issued by the
Bureau. Starting in 2003, the Bureau began releasing income and poverty
information from the American Community Survey, which produces data
independent from the Current Population Survey. Bureau officials
reported that for methodological and other reasons, estimates from the
Current Population Survey, in some cases, did not match estimates from
the American Community Survey, causing confusing press coverage. In
August 2004, when the Bureau released the two data sets at the same
time, the press release that accompanied the estimates explained why
the two sets of numbers might not match. (According to Bureau
officials, the plans to move the release date from September 2004 to
August 2004 were in place well before the actual release.) Going
forward, the Bureau plans to continue its practice of releasing the
American Community Survey data and the Income and Poverty Estimates
simultaneously around the last Thursday in August.
According to the Bureau, Several Factors Affected the Change in
Location of the 2003 Release:
According to a senior official we interviewed in the Bureau's Public
Information Office, the location of the 2003 Income and Poverty
Estimates news conference was changed from the National Press Club in
Washington, D.C., at the request of the Director of the Census Bureau,
to help raise awareness of the Bureau's new headquarters building,
which was under construction. The groundbreaking ceremony at the new
site on the Bureau's campus in Suitland, Maryland, had taken place
several weeks earlier, and a Bureau official reported the Director
wanted the media to see the improvements the Bureau was making at its
headquarters location, as well as to foster a spirit of good feeling,
and highlight how Bureau officials hoped that the new building would
help improve the morale of Bureau employees. The Bureau provided bus
service for attendees from the original location at the National Press
Club in downtown D.C., to Bureau headquarters in Suitland, a distance
of around 8 miles.
Additionally, according to the Bureau's Associate Director for
Strategic Planning and Innovation, the location of the news conference
is no longer as relevant as it once was because of changes in
technology. The 2003 news conference was broadcast in real time via the
Internet, and materials were made available on the Bureau's Web
site.[Footnote 12] The Associate Director for Strategic Planning and
Innovation noted that because of these advances and accommodations,
news media on-location attendance has declined over recent years. Yet,
overall media participation has increased via the availability of Web
casts, satellite-feed transmissions and telephone-audio access.
Consequently, the Suitland, Maryland headquarters is now the primary
location for this annual news conference.
Bureau Officials Made Key Decisions on Releasing Income and Poverty
Estimates:
Because the Bureau did not maintain a written record of the release
decision, a precise list of the personnel involved and time line of
events is unavailable. However, according to the Bureau officials we
interviewed, the following Bureau employees were involved in the
process for releasing the Income and Poverty Estimates in 2003:
* Director of the Census Bureau;
* Deputy Director/Chief Operating Officer;
* Chief of the Bureau's Housing and Household Economic Statistics
Division;
* Assistant Division Chief for Income, Poverty, and Health Statistics;
* Associate Director for Demographic Programs, now serving as the
Associate Director for Strategic Planning and Innovation;
* Associate Director for Communications;
* Staff from the Bureau's Housing and Household Economic Statistics
Division;
* Staff from the Bureau's Administrative and Customer Services
Division; and:
* Chief and Deputy Chief of the Bureau's Public Information Office.
Bureau officials said that prerelease access to the Income and Poverty
Estimates is tightly controlled because of the possible economic impact
of the data. They stated its contents are shared with staff on a need-
to-know basis, where only those individuals who are involved with
drafting the report or the accompanying press release have access to
the information. They noted further that key steps in preparing and
releasing the report included the following:
1. Program staff from the Bureau's Housing and Household Economic
Statistics Division drafted the Income and Poverty Estimates report.
2. A branch chief reviewed and approved the draft followed by the
Associate Division Chief, the Division Chief, and ultimately the
Associate Director for Demographic Programs, who reports to the Bureau
Director. These senior officials reviewed the report for such things as
clarity and presentation.
3. When the content of the report was finalized, the Bureau's Public
Information Office was sent a copy so it could draft a press release.
4. The final draft was sent to the Bureau's Administrative and Customer
Service Division, which designed the tables and figures, edited the
text, and prepared a camera-ready version of the report for printing.
According to Bureau officials, the Department of Commerce had only
limited access to information from the Income and Poverty Estimates
report before it was issued, and Commerce officials played no role in
the decision-making process surrounding its release. For example,
Commerce's Office of Public Affairs reviewed the press release that
accompanied the report and thus had access to some of the numbers as
well as the key findings in the report. However, the office did not
have access to any of the tables that are placed on the Internet.
(According to the Bureau, Commerce usually provides a "hook" for the
news media. In 2003, the press release was issued Friday, September 26,
and noted, on the first line, that the nation's poverty rate rose from
11.7 percent in 2001 to 12.1 percent in 2002.)[Footnote 13] Moreover,
the Bureau considers the press release part of the report and holds it
to the same standards for statistical quality as the report itself.
Additionally, according to Bureau officials, the Division Chief and the
Assistant Division Chief briefed the Director of the Census Bureau on
the report about a week before the September 26, 2003, press
conference. Commerce's Under Secretary for Economic Affairs was briefed
a day or two before the press conference and the Under Secretary's
staff were provided with the final report at that time. (The report was
also provided to the Council of Economic Advisers the afternoon before
the press conference.)
The Bureau Is Updating Its Practices for Releasing the Income and
Poverty Estimates:
The then Chief of the Census Bureau's Housing and Household Economic
Statistics Division told us the Bureau is updating its practices for
releasing the Income and Poverty Estimates. The official stated that
the Bureau was prompted to revisit the 1985 memo by the fact the memo
does not include all of the Bureau's long-standing data dissemination
practices; that some of the practices in the 1985 memo are obsolete
given the age of the guidance; and the rise of the Internet and other
technological advances have had an effect. The official added that the
process for releasing Income and Poverty Estimates has become more
formalized over time.
Bureau officials began drafting these revisions after the 2004 release.
In addition to updating the obsolete practices, Bureau officials stated
they planned to document the current practice of combining the Income
and Poverty Estimates release with the American Community Survey
release. The Bureau plans to issue its updated practices prior to the
next release of the Income and Poverty Estimates expected in August
2006.
Governmentwide Guidance Is Being Developed That May Improve Statistical
Agencies' Data Dissemination Practices:
Most of the 14 statistical agencies we reviewed reported general
adherence to NRC's guidance, important for (1) the wide dissemination
of data, and (2) maintaining a strong position of independence,
although there were some notable gaps. OMB, in concert with the
statistical agencies, has developed draft guidance on the release and
dissemination of statistical products that, according to OMB officials,
parallel NRC's guidance. To the extent it is comparable to NRC's
guidance and other widely accepted procedures for disseminating data,
the proposed OMB directive could promote more consistent adherence to
practices that promote broader dissemination of statistical data and
enhance the data's credibility.
Most Agencies' Data Dissemination Practices Generally Adhered to NRC's
Guidance for Disseminating Information and Maintaining Their
Independence:
According to NRC, statistical agencies must have "vigorous and well-
planned dissemination programs to get information into the hands of
users who need it on a timely basis." Attributes of a good
dissemination program include using a variety of mechanisms to inform
the widest possible audience about available data products and how to
acquire them. Agencies should also have arrangements for archiving the
information so that it is available for future use, as well as a
publications policy that describes, among other things, the types of
data products that will be made available, the frequency of their
release, and the audiences they serve.
NRC also notes that a statistical agency needs to be politically
independent; that is, it "must be impartial and avoid even the
appearance that its collection, analysis, and reporting processes might
be manipulated for political purposes. . . ." Elements of this practice
include having the authority for decisions associated with the scope,
content, and publication of the data, as well as the authority for the
selection and promotion of professional, operational, and technical
staff.
As shown in table 2, the data dissemination procedures of the 14
statistical agencies we reviewed included elements that were generally
aligned with NRC's guidance for the wide dissemination of data and
maintaining a strong position of independence.
Table 2: Most Statistical Agencies Reported That They Generally Adhered
to the National Research Council's Guidance for Releasing Information:
1;
Wide Dissemination of Data Elements: A variety of avenues for data
dissemination, chosen to reach as broad a public as reasonably
possible;
Number of Agencies That Reported Following NRC Guidance: [Empty];
Number of Agencies That Do Not Report Following NRC Guidance: [Empty].
Wide Dissemination of Data Elements: a) Regular communication of major
findings may be disseminated via an agency's Internet Web site,
government depository libraries, conference exhibits and programs,
newsletters and journals, e-mail address lists, and the media for
regular communication of major findings;
Number of Agencies That Reported Following NRC Guidance: : 14;
Number of Agencies That Do Not Report Following NRC Guidance: : --.
Wide Dissemination of Data Elements: b) The broadest possible audience
of potential users may be informed about available data products and
how to obtain them, and may include providing direct access to data on
the Internet, depositing data products in libraries, establishing a
network of data centers (such as the Census Bureau's state data
centers), holding exhibits and making presentations at conferences, and
maintaining lists of individuals and organizations to notify of new
data;
Number of Agencies That Reported Following NRC Guidance: : 14;
Number of Agencies That Do Not Report Following NRC Guidance: : --.
Wide Dissemination of Data Elements: c) Data releases are made
available in printed publications, on computer-readable media (e.g., CD-
ROM), and on the Internet;
Number of Agencies That Reported Following NRC Guidance: 2: 14;
Number of Agencies That Do Not Report Following NRC Guidance: 2: --.
2;
Wide Dissemination of Data Elements: Release of data in a variety of
formats (e.g., printed reports, various kinds of computer-readable data
files with careful, complete documentation);
Number of Agencies That Reported Following NRC Guidance: 14;
Number of Agencies That Do Not Report Following NRC Guidance: --.
3;
Wide Dissemination of Data Elements: Policies for the preservation of
data that guide what data to retain and how they are to be archived for
future secondary analysis;
Number of Agencies That Reported Following NRC Guidance: 14;
Number of Agencies That Do Not Report Following NRC Guidance: --.
4;
Wide Dissemination of Data Elements: An established publications policy
that describes, for a data collection program, the types of reports and
other data releases to be made available, the audience to be served,
and the frequency of release;
Number of Agencies That Reported Following NRC Guidance: [Empty];
Number of Agencies That Do Not Report Following NRC Guidance: [Empty].
Wide Dissemination of Data Elements: a) the types of reports to be made
available;
Number of Agencies That Reported Following NRC Guidance: : 9;
Number of Agencies That Do Not Report Following NRC Guidance: : 5.
Wide Dissemination of Data Elements: b) data releases to be made
available;
Number of Agencies That Reported Following NRC Guidance: : 11;
Number of Agencies That Do Not Report Following NRC Guidance: : 3.
Wide Dissemination of Data Elements: c) the audience to be served;
Number of Agencies That Reported Following NRC Guidance: : 7;
Number of Agencies That Do Not Report Following NRC Guidance: : 7.
Wide Dissemination of Data Elements: d) the frequency of release;
Number of Agencies That Reported Following NRC Guidance: 5: 8;
Number of Agencies That Do Not Report Following NRC Guidance: 5: 6.
5;
Wide Dissemination of Data Elements: Procedures for release of
information that preclude actual or perceived political interference.
In particular, the timing of the public release of data should be the
responsibility of the statistical agency;
Number of Agencies That Reported Following NRC Guidance: 12;
Number of Agencies That Do Not Report Following NRC Guidance: 2.
6;
Maintaining a Strong Position of Independence Elements: Dissemination
policies that foster regular, frequent release of major findings from
an agency's statistical programs to the public via the media, the
Internet, and other means;
Number of Agencies That Reported Following NRC Guidance: 8;
Number of Agencies That Do Not Report Following NRC Guidance: 6.
7;
Maintaining a Strong Position of Independence Elements: Adherence to
predetermined schedules in the public release of important statistical
indicators to prevent even the appearance of manipulation of release
dates for political purposes;
Number of Agencies That Reported Following NRC Guidance: [Empty];
Number of Agencies That Do Not Report Following NRC Guidance: [Empty].
Maintaining a Strong Position of Independence Elements: a)
Predetermined schedules exist for the public release of data (e.g.,
social and economic indicators and other statistical information), and;
Number of Agencies That Reported Following NRC Guidance: : 12;
Number of Agencies That Do Not Report Following NRC Guidance: : 2.
Maintaining a Strong Position of Independence Elements: b) When an
agency modifies a customary release schedule, this is done for
statistical purposes, and the agency announces and explains the change
in advance;
Number of Agencies That Reported Following NRC Guidance: 8: 5;
Number of Agencies That Do Not Report Following NRC Guidance: 8: 9.
8;
Maintaining a Strong Position of Independence Elements: Recognition by
policy officials outside the statistical agency of the agency's
authority to release statistical information without prior clearance by
department policy officials;
Number of Agencies That Reported Following NRC Guidance: 4;
Number of Agencies That Do Not Report Following NRC Guidance: 10.
9;
Maintaining a Strong Position of Independence Elements: Authority for
statistical agency heads and qualified staff to speak about the
agency's statistics before Congress, with congressional staff, and
before public bodies;
Number of Agencies That Reported Following NRC Guidance: 14;
Number of Agencies That Do Not Report Following NRC Guidance: -.
10;
Maintaining a Strong Position of Independence Elements: Maintenance of
a clear distinction between statistical information and policy
interpretations of such information by the President, the secretary of
the department, or others in the executive branch;
Number of Agencies That Reported Following NRC Guidance: 12;
Number of Agencies That Do Not Report Following NRC Guidance: 2.
11;
Maintaining a Strong Position of Independence Elements: Authority for
professional decisions over the scope, content, and frequency of data
compiled, analyzed, or published;
Number of Agencies That Reported Following NRC Guidance: 12;
Number of Agencies That Do Not Report Following NRC Guidance: 2.
12;
Maintaining a Strong Position of Independence Elements: Authority for
selection and promotion of professional, technical, and operational
staff;
Number of Agencies That Reported Following NRC Guidance: 5;
Number of Agencies That Do Not Report Following NRC Guidance: 9.
13;
Maintaining a Strong Position of Independence Elements: Authority to
ensure that information technology systems for data processing and
analysis securely maintain the integrity and confidentiality of data
and reliably support timely and accurate production of key statistics;
Number of Agencies That Reported Following NRC Guidance: [Empty];
Number of Agencies That Do Not Report Following NRC Guidance: [Empty].
Maintaining a Strong Position of Independence Elements: a) Statistical
agency has authority to secure information technology systems for data
processing and analysis;
Number of Agencies That Reported Following NRC Guidance: : 8;
Number of Agencies That Do Not Report Following NRC Guidance: : 6.
Maintaining a Strong Position of Independence Elements: b) Statistical
agency controls access to records of individual respondents by policy,
program, or regulatory agencies;
Number of Agencies That Reported Following NRC Guidance: Number of
Agencies That Reported Following NRC Guidance: 12;
Number of Agencies That Do Not Report Following NRC Guidance: Number of
Agencies That Do Not Report Following NRC Guidance: 2.
Source: GAO analysis.
[End of table]
Twelve or more of the agencies reported having data dissemination
practices possessing four of the five elements related to the wide
dissemination of data. All 14 agencies reported their data
dissemination practices followed NRC's guidance for (1) having multiple
avenues for disseminating data, (2) releasing data in a variety of
formats, and (3) having policies to guide what data should be preserved
and how it should be archived. Similarly, 12 or more of the agencies'
dissemination practices had characteristics associated with five of the
eight elements corresponding to maintaining a strong position of
independence. These elements include (1) adherence to predetermined
data release schedules, and (2) authority to make decisions over the
scope, content, and frequency of the data compiled, analyzed, or
published.
A greater number of agencies' data dissemination practices lacked
certain elements important for maintaining a strong position of
independence. An example of one of these elements is NRC's guidance
suggesting statistical agencies should have the "authority to release
statistical information and accompanying materials (including press
releases) without prior clearance by department policy officials" so
there is "no opportunity for or perception of political manipulation of
any of the information."
However, 10 of the 14 selected agencies reported varying degrees of
clearance required by department officials. For example, at 2 agencies,
the department rather than the statistical agency releases statistical
information. Other agencies have the authority to release statistical
information, except for press releases, without departmental clearance,
although in some cases, the department's clearance process is limited
to reviewing the grammar, punctuation, and other editorial aspects.
(Among the agencies in our review, 11 agencies use press releases; 1 of
these 11 agencies first publishes data from all of its major programs
via a press release; and the 3 remaining agencies reported they do not
use press releases as a vehicle to disseminate data.) With other
agencies the clearance process is more involved. For example, one
agency said it summarizes the data for the press release making sure it
is fair and complete, while officials at the departmental level might
insert comments from the cabinet secretary into the release. Further, 6
of the 14 agencies lacked dissemination policies that promote the
regular and frequent release of major findings from an agency's
statistical program.
As for the Bureau, officials reported that their agency generally
adhered to NRC's recommended guidelines. A notable gap was that Bureau
officials did not report adhering to announcing and explaining
modifications to a customary release schedule in advance (7b in table
2).[Footnote 14] Bureau officials also lacked the authority to release
statistical information and accompanying materials (including press
releases) without prior clearance by department policy officials (8 in
table 2). Also, while the Bureau's established publications policy
describes the frequency of release of data collection programs, the
Bureau reported the policy does not describe the types of reports to be
made available, the data releases to be made available, or the audience
to be served (4a-c in table 2).
OMB Is Developing Governmentwide Data Dissemination Guidance That Could
Help Strengthen the Credibility of Statistical Information:
OMB has been working with the federal Interagency Council on
Statistical Policy to develop guidance for the release and
dissemination of statistical products. According to OMB officials, the
guidance is intended to help ensure statistical products are policy-
neutral, timely, and accurate. OMB officials told us their directive is
similar to the NRC's recommended practices, as well as to OMB's
Statistical Policy Directive Number 3, which applies only to the 38
market-sensitive principal economic indicators produced by the
Departments of Agriculture, Commerce, Labor, and Treasury, as well as
the Federal Reserve Board. However, OMB officials told us this new
directive will not be as stringent as Statistical Policy Directive
Number 3, because the data covered by the directive are released less
frequently than the principal economic indicators, and the data are not
considered to be market-sensitive. OMB expects to issue the directive
for public comment in the spring of 2006.
To the extent OMB's dissemination directive appropriately addresses the
principles underlying NRC's guidance and Statistical Policy Directive
Number 3, the directive could enhance the quality and credibility of
federal statistical data, in part, by replacing the patchwork of agency-
specific guidance with a more transparent, commonly accepted, and
consistently applied framework for disseminating data. For example,
OMB's directive could help promote more consistent adherence to key
data release practices such as the wide dissemination of data and
maintaining an agency's independent position.
As noted in the previous section, the dissemination procedures at
several statistical agencies we examined lacked elements important for
these practices, including (1) authority to release statistical
information without prior clearance by department policy officials, (2)
data dissemination policies that foster the frequent release of major
findings from an agency's statistical programs, and (3) an established
publications policy that describes the types of reports and other data
releases to be made available. As a result, their data products could
be better protected, with the directive, from the appearance of, or
actual political involvement. More specifically, OMB's new directive
could address how best to address the gaps that exist between agencies'
data dissemination practices on the one hand, and NRC's guidance on the
other.
As OMB moves forward with its new directive, our interviews with OMB
and statistical agency officials, as well as our past work on data
quality guidance and internal control standards, identified the
following questions that will be important for OMB's dissemination
directive to consider:
Coverage: What will be covered by the directive?--principal statistical
agencies only?--the statistical functions of all agencies?--or only
statistical products? It will be important for OMB's directive to
clearly define what it does and does not cover so that both statistical
agencies and their parent organizations share the same understanding of
their respective authorities, and help ensure dissemination procedures
are consistently implemented. Certain roles, responsibilities, and
processes need to be clarified as well. Indeed, officials at two
statistical agencies we spoke with said there is ambiguity as to
whether a statistical press release is a statistical product and if so,
whether statistical agencies can issue them with or without first
getting releases cleared at the departmental level. Additionally, OMB
has issued a number of guidelines, directives, and standards on federal
statistics. Are there any gaps and overlaps among them, and can they be
better integrated?
Documentation: To what extent, and how, should agencies document their
data dissemination procedures and policies, and how often should they
be reviewed and updated? The agencies we examined did not always
document their processes for disseminating statistical data, relying
instead on professional practice. However, as NRC points out: "Although
a long-standing culture of data quality contributes to professional
practice, an agency should also seek to develop and document standards
through an explicit process."
Moreover, documented guidance would lend more transparency to the data
dissemination process, and thus provide a basis for agencies to explain
their dissemination decisions to policy makers, news media, and the
public. Indeed, an OMB official told us that Statistical Policy
Directive Number 3 is a useful tool for explaining to high level policy
officials the procedures agencies must follow to maintain the integrity
of the data, and why the officials may not access principal economic
data before it is released to the public nor comment on it until after
its release. Documented guidance could also help ensure continuity in
the face of employee turnover.
The importance of documenting agencies' data dissemination practices
can be seen in the Bureau's experience in releasing data from the 2000
Census on the homeless and others without conventional housing, when
the Bureau was criticized for shifting its position on reporting
components of this population. In our 2003 report, we noted that
although the Bureau's decision stemmed from its concerns over the
reliability of the underlying data, the Bureau's lack of documented,
clear, transparent, and consistently applied guidance governing the
release of data from the 2000 Census hampered the Bureau in explaining
its actions. Had such guidance been in place, it could have helped the
Bureau be more accountable and consistent in its dealings with the
public, and helped to ensure that the Bureau's decisions both were, and
appeared to be, totally objective.[Footnote 15]
Flexibility: How much leeway should agencies have in implementing OMB's
directive? Agency officials we spoke with noted the different missions
of the various statistical agencies and cautioned against a one-size-
fits-all approach. As a result, it might not be practical to require
all agencies to meet predetermined release dates because it could lead
to additional workload burdens and staffing issues.
Monitoring: How will OMB ensure agencies comply with its directive?
Indeed, the effectiveness of the policies and procedures laid out in
OMB's directive will rest in large part on the extent to which agencies
and their parent departments adhere to them. Related questions include
whether there should be a regular assessment of agencies' compliance,
and if so, how often should it occur, and whether this should be done
by OMB, or by the agencies through a self-assessment.
Posting Data: Should agencies' dissemination policies include written
guidance for releasing information via specific channels? Indeed,
although NRC's guidance calls on agencies to disseminate data using a
variety of outlets so that the information reaches as wide an audience
as possible, should agencies also have a standard set of conduits where
the public will know an agency's data will always be available? Such
conduits might include, among others, an agency's Web site. Because all
of an agency's data products would be, at a minimum, available from a
central point of access, it could help strengthen an agency's
credibility because the public would always know where to find it.
Conclusions:
A key lesson learned from the Bureau's experience is the importance of
fully documented, specific practices for maintaining the integrity of
data products, and by extension, the credibility of the agencies that
release them. Thus, as the Bureau updates its practices for releasing
the Income and Poverty Estimates, it will be important for the Bureau
to more thoroughly document its dissemination procedures so they are
clear to the public.
Further, OMB's efforts to develop governmentwide guidance on data
dissemination is a positive step toward enhancing the credibility of
federal statistical data, especially to the extent the directive
mirrors NRC's guidance and Statistical Policy Directive Number 3, as it
would replace each statistical agency's procedures with a more
transparent, commonly accepted, and consistently applied framework for
disseminating information. As OMB works to complete its directive, it
will be important for it to pay particular attention to those elements
dealing with the wide dissemination of data and maintaining a strong
position of independence that, our survey of statistical agencies
suggests can be adhered to by a greater number of agencies. Likewise,
OMB should also consider other aspects of agencies' data dissemination
efforts that could make its directive more comprehensive.
Recommendations for Executive Action:
To help improve the Bureau's data dissemination practices and thus
enhance the agency's actual and perceived position of independence, we
recommend that the Secretary of Commerce direct the Bureau to, as part
of its efforts to update its practices for releasing the Income and
Poverty Estimates, fully document its key data dissemination practices
for releasing the Income and Poverty Estimates.
Further, to help improve governmentwide data dissemination practices
that would further safeguard the integrity of federal statistical data,
we recommend that the Director of OMB ensure his agency, in completing
its draft directive on the release of federal statistical products,
considers whether and how to address areas where our survey indicates
there are gaps between NRC's existing guidance and agencies' practices.
These areas include the extent to which agencies should have (1) full
authority to release statistical information without prior clearance by
their respective departments, (2) data dissemination policies that
foster the frequent release of major findings from agency's statistical
programs, and (3) an established publications policy describing the
types of reports and other releases an agency has available.
We are also recommending that the Director of OMB direct his agency to
include in its directive additional elements and characteristics
important for agencies' data dissemination practices, including (1)
clear definitions of what is, and what is not covered by the directive,
(2) the extent to which agencies should document their data
dissemination guidance and how often the guidance should be reviewed,
(3) the amount of flexibility agencies have in implementing OMB's
guidance, (4) procedures for monitoring agencies' adherence to its
directive, and (5) the feasibility of requiring agencies to distribute
data products through a standard set of channels as well as through
other outlets as appropriate, so that the public will always know at
least one source it can turn to and obtain agency data.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, Commerce neither agreed
nor disagreed with our recommendation for the Bureau to fully document
its key data dissemination practices for releasing the Income and
Poverty Estimates. However, Commerce reiterated the point we made in
our report that the Bureau is updating its practices for releasing the
Income and Poverty Estimates. Commerce noted that the updated document-
-which details the dissemination practices for the Income and Poverty
Estimates--is under review. The Bureau plans to issue it prior to the
next release of the Income and Poverty Estimates expected in August
2006. Commerce also provided some technical corrections and suggestions
where additional context might be needed, and we revised the report to
reflect these comments as appropriate. Commerce's comments are
reprinted in their entirety in appendix II.
The Director of OMB did not have comments on our recommendations to
them. However, OMB officials provided suggestions for technical
corrections and we revised the report to reflect these suggestions as
appropriate.
As agreed with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
from its issue date. At that time, we will send copies of the report to
interested congressional committees, the Director of the U.S. Census
Bureau, and the Director of the Office of Management and Budget. Copies
will be made available to others on request. This report will also be
available at no charge on GAO's home page at [Hyperlink,
http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-6806 or farrellb@gao.gov. Contact points for
our Office of Congressional Relations and Public Affairs can be found
on the last page of this report. GAO staff that made major
contributions to this report are listed in appendix III.
Signed by:
Brenda S. Farrell:
Acting Director Strategic Issues:
[End of section]
Appendix I: Scope and Methodology:
To address the extent to which the U.S. Bureau of the Census (Bureau)
adhered to its dissemination practices for the release of the 2003
annual Income and Poverty Estimates and subsequent releases we asked
Bureau officials (in the Housing and Household Economic Statistics
Division and the Bureau's Public Information Office, among others) to
identify the Bureau and Department of Commerce officials who
participated in the data dissemination decisions, and interviewed the
identified officials to determine their role in the decision-making
process, and whether they had prerelease access to the information. We
compared their actions to the Bureau's data dissemination practices.
The dissemination process includes the steps from approval of the
report content up to and including public distribution of the report.
Some of these practices are documented in the Bureau's Policy and
Procedures Manual, while others are undocumented practices that we
identified by interviewing cognizant Bureau officials.
Because written records of key activities related to the release (e.g.
e-mails, meeting agendas and notes) were either not retained or never
created, much of our reconstruction of the release was based on
interviews with the officials involved. We interviewed many of these
officials both as a group (by department) and individually to obtain as
complete a picture of the events as possible, and corroborated the
information we received from the various parties involved.
To assess the extent to which the Bureau and other federal statistical
agencies[Footnote 16] followed data dissemination practices that the
National Academy of Sciences' National Research Council (NRC)
recommended in its 2005 report, Principles and Practices for a Federal
Statistical Agency, we surveyed officials at 14 federal statistical
agencies. (NRC prepared the report to assist statistical agencies in
making their products as sound as possible.)
Specifically, we surveyed officials at the Bureau, and 13 additional
federal statistical agencies to collect information on the procedures
they followed when releasing data. These 14 agencies comprise the
Interagency Council on Statistical Policy, a body that coordinates
federal statistical work and advises Office of Management and Budget
(OMB) on statistical matters. The 14 agencies are:
1. Bureau of Economic Analysis, U.S. Department of Commerce:
2. Bureau of Justice Statistics, U.S. Department of Justice:
3. Bureau of Labor Statistics, U.S. Department of Labor:
4. Bureau of Transportation Statistics, U.S. Department of
Transportation:
5. Bureau of the Census, U.S. Department of Commerce:
6. Economic Research Service, U.S. Department of Agriculture:
7. Energy Information Administration, U.S. Department of Energy:
8. National Agricultural Statistics Service, U.S. Department of
Agriculture:
9. National Center for Education Statistics, U.S. Department of
Education:
10. National Center for Health Statistics, U.S. Department of Health
and Human Services:
11. Office of Environmental Information, Environmental Protection
Agency:
12. Office of Research, Evaluation, and Statistics, Social Security
Administration:
13. Science Resources Statistics Division, National Science Foundation:
14. Statistics of Income Division, Internal Revenue Service, U.S.
Department of the Treasury:
In surveying the agencies, we reviewed relevant documents such as
agency policy manuals, and interviewed key officials who included,
depending on the agency, top management officials, chief statisticians,
as well as management staff from program, communications, or public
affairs offices.
We compared the information they provided us to certain practices that
the NRC has determined are important to federal statistical agencies in
the successful conduct of their missions. Specifically, we focused on
two NRC practices (1) wide dissemination of data, and (2) a strong
position of independence, because the 13 guidelines or elements
associated with these two practices are particularly important for data
dissemination. The first practice, the wide dissemination of data, is
associated with the mechanics of making the information available to
the public, including the media for releasing the information, as well
as how it is formatted and archived. The elements of the second
practice, a strong position of independence, are essential for
maintaining the credibility of statistical agencies, as well as for
providing an unimpeded flow of information to data users.
To obtain a broader perspective on the governmentwide framework for
helping to ensure data quality, we also interviewed OMB officials about
OMB's role in coordinating and overseeing the data dissemination
activities and reviewed appropriate OMB documents such as Statistical
Policy Directive Number 2 and Number 3.
We conducted our work between March 2005 and April 2006 in accordance
with generally accepted government auditing standards.
[End of section]
Appendix II: Comments from the Department of Commerce:
The Deputy Secretary Of Commerce:
Washington, D.C. 20230:
May 19, 2006:
Ms. Brenda S. Farrell:
Acting Director:
Strategic Issues:
United States Government Accountability Office:
Washington, DC 20548:
Dear Ms. Farrell:
The U.S. Department of Commerce appreciates the opportunity to comment
on the United States Government Accountability Office's draft report
entitled Expanded Use of Key Dissemination Practices Would Further
Safeguard the Integrity of Federal Statistical Data. The Department's
comments on this report are enclosed.
Sincerely,
Signed by:
David A. Sampson:
Enclosure:
U.S. Department of Commerce Comments on the United States Government
Accountability Office Draft Report Entitled Expanded Use of Key
Dissemination Practices Would Further Safeguard the Integrity of
Federal Statistical Data (GAO-06-607):
comments on Recommendation:
On page 6, the draft report makes the following recommendation:
We are recommending that the Secretary of Commerce ensure that the
Bureau, as part of its efforts to update its practices for releasing
the Income and Poverty Estimates, fully document those practices.
As the draft report points out, the U.S. Census Bureau has drafted an
updated document that describes, in full detail, the data dissemination
practices for its Income and Poverty Estimates. That document is
currently undergoing review and is targeted to be finalized before this
year's planned August release of 2005 income and poverty estimates.
Other Comments:
On page 4, the draft report states that the planned release date was
changed from September 23 to September 26 ". . . because of delays in
producing a companion report on supplemental measures of expenditures,
consumption, and poverty." The draft report also discusses reasons for
the delay on page 14. While the production of this new companion report
was a factor in this decision,
there were other changes in the reports released September 2003 that
played a role. For completeness, we suggest that these other factors
should be mentioned or at least footnoted, as follows:
1. The September 2003 reports were the first income and poverty reports
to include tables and text based on the new Census 2000-consistent race
groups.
2. They were also the first to reflect Census 2000-consistent
industries and occupations.
3. New tables and analysis on alternative income definitions were added
to the income report released in September 2003. All of these changes,
along with the production of the companion report, factored into the
decision to delay the release.
On page 11, under step 7, there should be an explanatory note that
states "For the last three years, the media have received the report as
if it were an economic indicator. After media are seated in the
auditorium and the procedures are announced, the report is then
distributed at approximately 10:10 A.M. The Internet link is also
opened at 10:10 A.M."
On page 12, we suggest an addition to the end of the last sentence in
the first paragraph, so that the end of the sentence would read ". . .
the Bureau followed the steps in the 1985 memo in the 2003, 2004 and
2005 releases, with the exception of the release time as previously
described."
On page 14, the draft report states, ". . . while the Bureau had
completed its review of the latter report, a steering committee still
needed to review the report on supplemental measures." We suggest this
wording: ". . . while the Bureau had completed its review of the latter
report, all the members of a steering committee still needed to review
the report on supplemental measures."
Also on page 14, the draft report states, "Because of the hold-up in
clearing the supplemental measures report. We suggest this wording:
`Because of the additional time required to clear the supplemental
measures report .
We also suggest the following wording for the last sentence in that
paragraph: "Consequently, the Bureau's Associate Director for
Communications, with the Director's consent, scheduled Friday,
September 26, 2003, as the first available workday for the release of
the Income and Poverty Estimates, and both reports were issued on that
date."
On page 15, we suggest the following wording for the next-to-last
sentence in the first paragraph: "Additionally, Bureau officials said
that because of the Internet and cable television, the news cycle is no
longer viewed as a cycle and has instead become a 2417 operation. Many
of the questions come the same day as the data are released."
In the next paragraph, we suggest that you delete "responsible for the
2004 release" from the second sentence.
Also, in the same paragraph, it is incorrect to say that "until 2003,
the Income and Poverty Estimates (which come from the Bureau's Current
Population Survey) was the only source of income and poverty
information issued by the Bureau." The Census Bureau issues income and
poverty data from several sources other than the Current Population
Survey (CPS) and the American Community Survey (ACS), including the
decennial census, the Survey of Income and Program Participation, and
our Small Area Income and Poverty Estimates Program. Additionally, ACS
income and poverty estimates for the Nation, states, and large cities
and counties were released in the summer of 2001 and 2002. The
difference in 2003 was that this was the first year in which there was
a significant amount of press coverage that confused the ACS and CPS
estimates.
In the same paragraph, in discussing the change in release date from
September to August 2004, you should note that the plans to make this
change were in place well before the actual release. There was a
memorandum sent by the Census Bureau Director to Department of Commerce
Census Bureau's intention to inform the Office of Management and Budget
of this change.
Under Secretary Kathleen Cooper on January 13, 2004, titled "Income and
Poverty Reports for Reference Year 2003," that outlined that year's
release plans, the reasons behind them, and the:
On page 16, first paragraph, change "central location" to "headquarters
location," and we suggest this wording for the end of the last
paragraph: "Census Bureau officials noted that because of these
advances and accommodations, news media on-location attendance has
declined over recent years. Yet, overall media participation has
increased via the availability of Webcasts, satellite-feed
transmissions, and telephone-audio access. Consequently, the Suitland,
Maryland, headquarters is now the primary location for this annual news
conference."
On page 17, change the last bullet to "Chief and Deputy Chief of the
Bureau's Public Information Office," and in the next paragraph, change
"prerelease" to "pre-release."
On page 18, we suggest deleting this sentence: "According to the
Bureau, Commerce usually provides a "hook" for the news media."
On page 24, it is not true that the Census Bureau did not report
modifications to the customary release schedule. In 2004, as noted
previously, there was a memorandum written by the Census Bureau
Director in January 2004 that described the rationale for the change.
There was a Note to Correspondents issued on April 5, 2004, that
informed the media and stakeholders about the change. There was also a
Media Advisory released on August 19, 2004, that highlighted the fact
that ACS and CPS data were being released concurrently for the first
time.
We also believe that the Census Bureau has the authority to ensure that
technology systems securely maintain the integrity and confidentiality
of data and that these systems reliably support timely and accurate
productions of key statistics.
[End of section]
Appendix III: GAO Contact and Acknowledgments:
GAO Contact:
Brenda S. Farrell, (202) 512-6806:
Acknowledgments:
In addition to the individual named above, Robert Goldenkoff, Assistant
Director, as well as Timothy Wexler, April Thompson, Robert Parker, Jay
Smale, Michael Volpe, Andrea Levine, and Amy Rosewarne made key
contributions to this report.
[End of section]
Related GAO Reports:
Quality of Federal Data:
Information Quality Act: National Agricultural Statistics Service
Implements First Steps, but Documentation of Census of Agriculture
Could Be Improved. GAO-05-644. Washington, D.C.: September 23, 2005.
Data Mining: Agencies Have Taken Key Steps to Protect Privacy in
Selected Efforts, but Significant Compliance Issues Remain. GAO-05-866.
Washington, D.C.: August 15, 2005.
Data Quality: Improvements to Count Correction Efforts Could Produce
More Accurate Census Data. GAO-05-463. Washington, D.C.: June 20, 2005.
Data Quality: Census Bureau Needs to Accelerate Efforts to Develop and
Implement Data Quality Review Standards. GAO-05-86. Washington, D.C.:
November 17, 2004.
Decennial Census: Methods for Collecting and Reporting Hispanic
Subgroup Data Need Refinement. GAO-03-228. Washington, D.C.: January
17, 2003.
Decennial Census: Methods for Collecting and Reporting Data on the
Homeless and Others without Conventional Housing Need Refinement. GAO-
03-227. Washington, D.C.: January 17, 2003.
2000 Census: Refinements to Full Count Review Program Could Improve
Future Data Quality. GAO-02-562. Washington, D.C.: July 3, 2002.
FOOTNOTES
[1] Margaret E. Martin, Miron L. Straf, and Constance F. Citro eds.,
Principles and Practices for a Federal Statistical Agency, (3rd ed.)
(Washington, D.C.: The National Academies Press, 2005). The National
Academy of Sciences organized the National Research Council in 1916 "to
associate the broad community of science and technology with the
Academy's purposes of furthering knowledge and advising the federal
government." According to Ms. Citro, the Principles and Practices for a
Federal Statistical Agency report is the signature product of the
Committee on National Statistics (which is a standing committee of the
National Research Council).
[2] The Census Bureau's American Community Survey is a monthly sample
survey of 250,000 households that is to replace the long-form
questionnaire used in prior decennial censuses.
[3] Consolidated Appropriations - Fiscal Year 2001, Pub. L. No. 106-
554, § 515, 114 Stat. 2763A-153 to 2763A-154 (2000) (44 U.S.C. § 3516
note).
[4] http://www.census.gov/quality/quality_guidelines.htm (downloaded
Oct. 3, 2003).
[5] The standard, Dissemination of Census and Survey Data Products,
became effective on May 19, 2005, so it did not apply to the 2003 or
2004 income and poverty releases. However, it covered the 2005 release
and will cover all future Income and Poverty Estimates releases.
[6] The Bureau created this document in response to a recommendation
contained in our report, GAO, Data Quality: Census Bureau Needs to
Accelerate Efforts to Develop and Implement Data Quality Review
Standards, GAO-05-86 (Washington, D.C.: Nov. 17, 2004).
[7] According to Bureau officials, in 2003, 2004 and 2005 the media
received the Income and Poverty report as if it were an economic
indicator. After media were seated in the auditorium and the procedures
were announced, the report was then distributed at approximately 10:10
AM. The Internet link was also opened at 10:10 AM.
[8] The Housing and Household Economic Statistics Division is the unit
responsible for the Income and Poverty Estimates.
[9] Supplemental Measures of Material Well-Being: Expenditures,
Consumption, and Poverty 1998 and 2001 (Washington, D.C.: U.S.
Department of Commerce. Economics and Statistics Administration, U.S.
Census Bureau. September 2003). The report was a collaborative effort
by members of the Interagency Working Group on Alternative Measures of
Material Well-Being. The group was composed of members from the
following agencies: the Office of Management and Budget; the Bureau of
Labor Statistics in the U.S. Department of Labor; the Energy
Information Administration in the U.S. Department of Energy; and the
Census Bureau, Office of Policy Analysis and Bureau of Economic
Analysis in the U.S. Department of Commerce.
[10] The Bureau cited several other factors for changing the date of
the 2003 Income and Poverty Estimates release: (1) the September 2003
report was the first income and poverty report to include tables and
text based on the new Census 2000-consistent race groups; (2) the
September 2003 report was also the first to reflect Census 2000-
consistent industries and occupations; and (3) new tables and analysis
on alternative income definitions were added to the income report
released in September 2003.
[11] http://www.census.gov/Press-Release/www/releases/archives/
health_care_insurance/ 000526.html (downloaded Apr. 20, 2006).
[12] http://www.census.gov/Press-Release/www/2003/IncomePoverty.html
(downloaded Apr. 20, 2006).
[13] http://www.census.gov/Press-Release/www/releases/archives/
income_wealth/001371.html (downloaded Apr. 20, 2006).
[14] In its comments on a draft of this report, the Department of
Commerce noted that prior to the 2004 release of the Income and Poverty
Estimates the Census Bureau Director wrote a memorandum that described
the rationale for the change to the customary release schedule.
Although the Bureau announced the rationale for the schedule change in
this instance, in our survey of agencies' data dissemination
procedures, Bureau officials told us that the Bureau does not always
adhere to the practice of announcing and explaining modifications to a
customary release schedule in advance.
[15] GAO, Decennial Census: Methods for Collecting and Reporting Data
on the Homeless and Others without Conventional Housing Need
Refinement, GAO-03-227 (Washington, D.C.: Jan. 17, 2003).
[16] A federal statistical agency's principal function is the
compilation and analysis of data and the dissemination of information
for statistical purposes.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
