Bureau of Justice Statistics

Quality Guidelines Generally Followed for Police-Public Contact Surveys, but Opportunities Exist to Help Assure Agency Independence Gao ID: GAO-07-340 March 30, 2007

The Bureau of Justice Statistics (BJS), a statistical agency of the Department of Justice's Office of Justice Programs, produces a recurring national Police-Public Contact Survey documenting contacts between the police and the public, including instances involving the use or threat of force by police. BJS issues public reports and sometimes press releases from survey results. For reports and a press release issued from the 1999 and 2002 surveys (the most recent available), GAO reviewed (1) the extent to which BJS followed quality guidelines to ensure the accuracy and integrity of its survey-related products, and (2) factors that affected whether and how BJS followed available guidelines. GAO reviewed applicable federal data quality guidelines, policy and procedure documents, and interviewed current and former officials familiar with BJS.

BJS followed nearly all quality guidelines for its 1999 and 2002 Police-Public Contact Surveys. Specifically, for the four public reports issued from these surveys, BJS fully followed all data quality guidelines available for reviewing statistical information, obtaining the approval of key decision makers, and publicly disseminating information. These guidelines were issued by the National Research Council, Department of Justice, Justice's Office of Justice Programs, and BJS itself. GAO believes that because BJS followed these guidelines, proper steps were taken to help ensure the accuracy and integrity of the reports. BJS followed 7 of the 10 quality guidelines available for the one press release issued from its 1999 survey, but was not in a position to fully follow 3 other guidelines for reasons discussed below. Two key factors affected whether and how BJS followed quality guidelines. The first concerned different interpretations about certain guideline applicability. BJS considered its survey-related reports--but not its press releases--to be statistical products covered by the National Research Council's guidelines. BJS cited a lack of specificity in these guidelines, which did not specifically state that they were applicable to statistical agency press releases, as a basis for concluding that the survey press releases need not conform to guidelines for statistical products. We believe BJS's position was not unreasonable, and did not find fault with the agency. However, we determined nonetheless that the single press release issued from the 1999 survey was a statistical product, and therefore believe the council's guidelines appropriately applied. Second, certain noncareer appointees outside BJS may, in accordance with Justice Department policy, make decisions about the review, approval, and dissemination of press releases, and BJS press releases are jointly issued with the Justice Department, with input from its Office of Justice Programs. Both conditions can potentially affect BJS's independence. Owing to these conditions, BJS was not, in our view, in a position to meet 3 council quality guidelines related to statistical agency independence, including that it be able to issue statistical products without prior clearance, and control the scope and content of its products. Justice affirmed several of GAO's findings but disagreed with certain GAO conclusions about the applicability of guidelines to a press release. Justice's detailed comments and GAO's response are contained in the report.



GAO-07-340, Bureau of Justice Statistics: Quality Guidelines Generally Followed for Police-Public Contact Surveys, but Opportunities Exist to Help Assure Agency Independence This is the accessible text file for GAO report number GAO-07-340 entitled 'Bureau of Justice Statistics: Quality Guidelines Generally Followed for Police-Public Contact Surveys, but Opportunities Exist to Help Assure Agency Independence' which was released on April 30, 2007. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Requesters: United States Government Accountability Office: GAO: March 2007: Bureau Of Justice Statistics: Quality Guidelines Generally Followed for Police-Public Contact Surveys, but Opportunities Exist to Help Assure Agency Independence: GAO-07-340: GAO Highlights: Highlights of GAO-07-340, a report to congressional requesters Why GAO Did This Study: The Bureau of Justice Statistics (BJS), a statistical agency of the Department of Justice‘s Office of Justice Programs, produces a recurring national Police-Public Contact Survey documenting contacts between the police and the public, including instances involving the use or threat of force by police. BJS issues public reports and sometimes press releases from survey results. For reports and a press release issued from the 1999 and 2002 surveys (the most recent available), GAO reviewed (1) the extent to which BJS followed quality guidelines to ensure the accuracy and integrity of its survey-related products, and (2) factors that affected whether and how BJS followed available guidelines. GAO reviewed applicable federal data quality guidelines, policy and procedure documents, and interviewed current and former officials familiar with BJS. What GAO Found: BJS followed nearly all quality guidelines for its 1999 and 2002 Police- Public Contact Surveys. Specifically, for the four public reports issued from these surveys, BJS fully followed all data quality guidelines available for reviewing statistical information, obtaining the approval of key decision makers, and publicly disseminating information. These guidelines were issued by the National Research Council, Department of Justice, Justice‘s Office of Justice Programs, and BJS itself. GAO believes that because BJS followed these guidelines, proper steps were taken to help ensure the accuracy and integrity of the reports. BJS followed 7 of the 10 quality guidelines available for the one press release issued from its 1999 survey, but was not in a position to fully follow 3 other guidelines for reasons discussed below. Two key factors affected whether and how BJS followed quality guidelines. The first concerned different interpretations about certain guideline applicability. BJS considered its survey-related reports”but not its press releases”to be statistical products covered by the National Research Council‘s guidelines. BJS cited a lack of specificity in these guidelines, which did not specifically state that they were applicable to statistical agency press releases, as a basis for concluding that the survey press releases need not conform to guidelines for statistical products. We believe BJS‘s position was not unreasonable, and did not find fault with the agency. However, we determined nonetheless that the single press release issued from the 1999 survey was a statistical product, and therefore believe the council‘s guidelines appropriately applied. Second, certain noncareer appointees outside BJS may, in accordance with Justice Department policy, make decisions about the review, approval, and dissemination of press releases, and BJS press releases are jointly issued with the Justice Department, with input from its Office of Justice Programs. Both conditions can potentially affect BJS‘s independence. Owing to these conditions, BJS was not, in our view, in a position to meet 3 council quality guidelines related to statistical agency independence, including that it be able to issue statistical products without prior clearance, and control the scope and content of its products. Justice affirmed several of GAO‘s findings but disagreed with certain GAO conclusions about the applicability of guidelines to a press release. Justice‘s detailed comments and GAO‘s response are contained in the report. Table: Guidelines BJS Followed for Products Issued from 1999 through 2002 Police-Public Contact Surveys: Reports: Number of reports issued: 4; Reports: Number of guidelines available: Up to 23[A]; Reports: Number of guidelines fully followed: All; Press releases: Number of press releases issued: 1[B]; Press releases: Number of guidelines available: 10; Press releases: Number of guidelines fully followed: 7. Source: GAO analysis of agency data. [A] Total guidelines were 24 across the different points in time; 23 was maximum at any given time. [B] From 1999 survey information. [End of table] What GAO Recommends: In a May 2006 report, GAO recommended that the Office of Management and Budget (OMB) issue a directive to further safeguard the integrity of federal statistical data and improve guideline clarity. Such a directive could help address conflicting interpretations about the applicability of quality guidelines. Since OMB is still working on this directive, GAO makes no new recommendations in this report. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-340]. To view the full product, including the scope and methodology, click on the link above. For more information, contact Brian Lepore at (202) 512- 4523 or leporeb@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: BJS Followed Nearly All Available Quality Guidelines to Help Ensure Accuracy and Integrity of Products Issued from Police-Public Contact Surveys: Differing Views on Applicability of Guidelines Raised Questions about Clarity, while Roles of Certain Noncareer Appointees during Product Issuance Process Affected BJS's Independence: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: Applicable Quality Guidelines for Statistical Products Issued by the Bureau of Justice Statistics and Other Agencies: Appendix III: Press Release Issued in 2001 Based on BJS's 1999 Police- Public Contact Survey: Appendix IV: BJS Processes for the Review, Approval, and Dissemination of Police-Public Contact Survey Products: Appendix V: Changes in the Scope and Methodology of the Police-Public Contact Surveys: Appendix VI: Comments from the Department of Justice: Appendix VII: GAO Contact and Acknowledgments: Tables: Table 1: Public Products Issued Based on 1999 and 2002 Police-Public Contact Surveys: Table 2: Guidelines That BJS Fully Followed for the Reports Issued from Its 1999 and 2002 Police-Public Contact Surveys: Table 3: Noncareer Appointees' Roles in the Review, Approval, and Dissemination of BJS Press Releases: Table 4: Extent to Which BJS Followed Available Quality Guidelines for 1999 and 2002 Police-Public Survey Products: Table 5: Police-Public Contact Survey (PPCS) Methodologies for the Four Surveys: 1996, 1999, 2002, and 2005: Figures: Figure 1: Office of Justice Programs Organizational Structure: Figure 2: Review, Approval, and Dissemination Process for BJS Survey Reports: Figure 3: Review, Approval, and Dissemination Process for BJS Survey Press Releases: Abbreviations: BJS: Bureau of Justice Statistics: CATI: computer-assisted telephone interview: DOJ: Department of Justice: FBI: Federal Bureau of Investigation: NRC: National Research Council: OCOM: Office of Communications: OJP: Office of Justice Programs: OMB: Office of Management and Budget: PAPI: paper-and-pencil in-person face-to-face interview: United States Government Accountability Office: Washington, DC 20548: March 30, 2007: Congressional Requesters: The Bureau of Justice Statistics (BJS), a statistical agency within the Department of Justice's Office of Justice Programs, collects, analyzes, publishes, and disseminates information on crime, criminal offenders, victims of crime, and the operation of justice systems at all levels of government. A critical aspect of the agency's mission is to produce impartial, timely, and accurate statistics. An ongoing BJS data collection activity is the Police-Public Contact Survey--one of many statistical surveys the agency administers. This recurring, nationally representative survey of the public is designed to document contacts between the police and the public and the outcomes of those contacts, including instances involving the use of force or the threat of force by the police. BJS issues reports to the public based on the results of this survey, and these may be accompanied by a press release. To date, BJS has conducted the survey four times--a pilot survey was conducted in 1996, and more extensive surveys were done in 1999, 2002, and 2005.[Footnote 1] The Police-Public Contact Survey involves conducting interviews in person and by telephone with tens of thousands of individuals to obtain information about their encounters with police officers. The most frequently reported reason for such contacts has involved traffic stops. In this context, interviewers have gathered self-reported data from motorists on such factors as the reason for the traffic stop and what occurred after the traffic stop--for example, whether the individual was ticketed, handcuffed, or searched, and whether the police used or threatened to use force. In the spring of 2005, a disagreement arose between the then-director of BJS and an official outside of BJS but within the Department of Justice about the statistical content of a draft press release concerning the results of the 2002 Police-Public Contact Survey (which at the time was the most recent survey to have been completed), and no press release was ultimately issued from this survey. As a result of this incident, some members of Congress, the research community, and the media raised questions about the methods and procedures BJS follows in reviewing and approving reports and other products based on the agency's statistical information, and the extent of BJS's independence as a statistical agency. In light of the questions raised about BJS's ability to meet its self- stated mission as a statistical agency committed to maintaining public trust and confidence in its statistics, we reviewed what processes are in place at BJS to help the agency ensure the integrity and independence of its statistical studies. This report specifically addresses the following two questions for the 1999 and 2002 Police- Public Contact Surveys, the two surveys for which products had been issued as of February 2007: (1) To what extent did BJS follow guidelines to help ensure the accuracy and integrity of the review, approval, and dissemination of reports and press releases based on its surveys? (2) What key factors affected whether and how BJS followed available guidelines? To assess the extent to which BJS followed guidelines to help ensure the accuracy and integrity of its product issuance processes for the 1999 and 2002 Police-Public Contact Surveys, we reviewed quality guidelines pertaining to the review, approval, and dissemination of written products issued by the National Research Council (a private, nonprofit institution that is a component of the congressionally chartered National Academies), the Department of Justice, the Office of Justice Programs, and BJS itself. The guidelines that we reviewed covered the period between February 2001, when the first product based on the 1999 survey was issued, and June 2006, when the most recent product based on the 2002 survey was issued. We initially identified a total of 63 guidelines from the four organizations. Because the organizations had a number of similar guidelines, we eliminated the overlap by reducing the list to 24 nonduplicative guidelines. To determine the extent to which BJS followed each guideline, we developed a data collection instrument for recording information on whether BJS fully, partially, or did not at all follow the guideline. For BJS's 1999 and 2002 Police-Public Contact Survey reports and the one press release that was issued, we asked BJS to assess the extent to which it followed each of 15 guidelines, and provide us with documentary evidence supporting its assessment. We completed the instrument for the remaining 9 guidelines for which we already had sufficient information from BJS and Office of Justice Programs documents and interviews, and asked BJS to confirm or not confirm our assessments. Two GAO analysts reviewed BJS's responses and all available supporting documentary and testimonial evidence, and made a final determination on the extent to which we believed BJS followed each guideline. We provided our findings to BJS for review and comment. We also reviewed guidelines issued by the Office of Management and Budget (OMB), but did not specifically assess BJS's practice with respect to following OMB's guidelines, because OMB issued governmentwide policy and procedural guidance to federal agencies that called for agencies to develop their own agency- specific guidelines. To determine key factors that affected whether and how BJS followed applicable guidelines, we reviewed BJS procedural documents and interviewed both current and former officials involved in the review, approval, and dissemination of BJS products, including past and present BJS directors and assistant attorneys general within the Office of Justice Programs. In addition, we reviewed pertinent statutory provisions relating to the roles and responsibilities of officials with respect to BJS. It was beyond the scope of this review to address any personnel issues that may have arisen in connection with the disagreement over the content of the draft press release. Appendix I provides additional details on our scope and methodology. We conducted our work between April 2006 and February 2007 in accordance with generally accepted government auditing standards. In addition, in response to your interest regarding changes that have occurred in the scope and methodology of the Police-Public Contact Surveys between 1996 and 2005, and the reasons for any changes, appendix V provides detailed information on this issue. Results in Brief: For the four issued reports and the one issued press release based on the 1999 and 2002 Police-Public Contact Surveys, BJS followed nearly all quality guidelines related to product issuance developed by various organizations. These reports and press release were issued between 2001 and 2006, and are the most recent products available from the surveys. Specifically, for the four reports issued from these surveys, we found that BJS fully followed all product issuance guidelines of the National Research Council, the Department of Justice, the Office of Justice Programs, and BJS itself, pertaining to how agencies should review statistical information, obtain the approval of key decision makers, and publicly disseminate the information. Because BJS followed these guidelines, we believe it took proper steps to help ensure the accuracy and integrity of the processes associated with issuing public reports based on Police-Public Contact Surveys. BJS concurred with our analysis. With respect to the single press release issued based on the Police-Public Contact Survey, we found that BJS was in a position to fully follow 7 of 10 quality guidelines available for the one press release issued from the 1999 survey.[Footnote 2] BJS did not, in our view, fully follow 3 other guidelines pertaining to agency independence for reasons discussed below. Two key factors had an impact onaffected whether and how BJS followed quality guidelines during the review, approval, and dissemination processes followed for of the press releases that could beproducts issued from the 1999 and 2002 Police-Public Contact Surveys. First, BJS believed certain quality guidelines, including those issued by the National Research Council, applied to its survey-related reports, but not to press releases. This was did not apply to survey-related press releases because it BJS considered its reports to be statistical products covered by the guidelines, and did not consider thempress releases not to be statistical products, and therefore not covered by the quality guidelines. BJS cited a lack of specificity in the National Research Council's guidelines available at the time as a basis for this conclusion. We believe, however, that while we believe that BJS's conclusion was not unreasonable, our analysis of the press release issued led us to conclude--based on its content as opposed to its label as a press release, and regardless of the procedures used to develop and issue it--that it was primarily a statistical product. there was nonetheless sufficient evidence to indicate that the press release issued from the 1999 survey (a release was not issued from the 2002 survey) was a statistical product and that in our view, it was therefore appropriate to apply the council's guidelines to it. We acknowledge that, given the lack of specificity that existed at the time in the that the National Research Council's councils printed guidelines did not specifically state that they covered press releases based on the statistical reports of statistical agencies, , BJS could reach a different conclusion, and therefore we do not fault BJS for concluding that the 2001 press release based on the 1999 Police-Public Contact Survey was exempt from the guidelines. Indeed, we noted in our May 2006 report on data quality that 2 of 14 statistical agencies we surveyed stated that there was ambiguity as to whether a statistical press release was a statistical product. BJS was among the 14 statistical agencies surveyed, but it was not 1of the 2 agencies reporting ambiguity in whether a statistical press release was a statistical product. The second factor that has had an impact on affected whether and how BJS followed guidelines in its product issuance process concerned is the role that certain noncareer appointees outside BJS can play in the product issuance process., in part, as a matter of Justice Department policy--which in turn affects BJSs position as an independent statistical agency. In particular, the Assistant Attorney General in the Office of Justice Programs has the ability to certain noncareer appointees outside of BJS, including the Assistant Attorney General in the Office of Justice Programs, have the ability under Department of Justice policy to become involved in the review, approval, and dissemination of press releases; and press releases are issued jointly by BJS, the Department of Justice, and the Office of Justice Programs. Owing to these circumstances, BJS was not in a position to follow the National Research Council guidelines recommending that statistical agencies issue statistical products without prior clearance; and control the scope, content, and timing of their products--issues relating to BJS's independence as a statistical agency. To address conflicting interpretations of certain quality guidelines, we recommended in a May 2006 report on the quality of federal data that OMB issue a directive that, among other things, would provide clear definitions about what guidelines cover.[Footnote 3] As of January 2007, according to an OMB analyst assigned to the effort, OMB was still working on this directive. Because our May 2006 recommendation to OMB has not yet been implemented, we are not making any new recommendations in this report. We continue to believe, however, that implementing our recommendation could help safeguard the integrity of federal statistical data. We also believe it would reduce the likelihood that the type of disagreement discussed in this report would recur, and help assure the independence of BJS as a statistical agency. We provided a draft of this report to the Department of Justice and received written comments from the Justice Department's Office of Justice Programs' Assistant Attorney General. The Assistant Attorney General affirmed several of our findings and agreed that a need exists for clear definitions about what constitutes a statistical product and that the federal statistical community would benefit from clarity in this area. The agency disagreed, however, with our characterization of the 2001 Police-Public Contact Survey press release as a statistical product, and therefore did not believe that the National Research Council's or BJS's own quality guidelines were applicable to it. We maintain that we made a sound decision in applying these guidelines to the Police-Public Contact Survey press release because we felt that the content of the press release, which was almost entirely statistical in nature, was a more important determinant of whether or not it was a statistical product than the label attached to it. The Assistant Attorney General's comments appear in appendix VI and our detailed response to these comments is contained in the report. Background: Overview of BJS Mission and Organization: BJS was established by the Justice Systems Improvement Act of 1979.[Footnote 4] In 1995, OMB identified BJS as one of 10 principal statistical agencies within the federal government. As defined by OMB,[Footnote 5] the statistical activities of statistical agencies include the planning of statistical surveys and studies; and the collection, processing, or tabulation of statistical data for publication, dissemination, research, analysis, or program management and evaluation. BJS publishes annual data on criminal victimization, populations under correctional supervision, and federal criminal offenders and case processing. It provides periodic data series on the administration of law enforcement agencies and correctional facilities, prosecutorial practices and policies, state court case processing, felony convictions, the characteristics of correctional populations, criminal justice expenditure and employment, civil case processing in state courts, and special studies on other criminal justice topics. BJS is organizationally located within the Department of Justice's Office of Justice Programs (see fig. 1). The highest-ranking executives of BJS (BJS Director) and the department's Office of Justice Programs (Assistant Attorney General) are both noncareer officials appointed by the President and confirmed by the Senate. Within BJS, only the Director is a noncareer appointee. Figure 1: Office of Justice Programs Organizational Structure: [See PDF for image] Source: BJS. [End of figure] The BJS Police-Public Contact Survey: BJS initiated the Police-Public Contact Survey pursuant to a mandate in the Violent Crime Control and Law Enforcement Act of 1994,[Footnote 6] which required the Attorney General to collect information on the use of excessive force by law enforcement officers. The data were to be used only for research or statistical purposes and were not to contain any information that could reveal the identity of the victim or any law enforcement officer. BJS fielded its first pilot survey in 1996 with the goal of better understanding the types and frequency of contacts between the police and the public, and the conditions under which force may be threatened or used. The pilot survey consisted of 6,421 respondents. The three subsequent surveys (in 1999, 2002, and 2005) consisted of 80,543, 76,910, and 63,943 respondents, respectively. Multiple reports and press releases may be issued in connection with any of the surveys. The years in which reports and a single press release associated with the 1999 and 2002 surveys were issued are shown in table 1. Table 1: Public Products Issued Based on 1999 and 2002 Public-Police Contact Surveys: Date of survey: 1999; Years reports issued: 2001, 2002[A,B]; Year press release issued: 2001[E]. Date of survey: 2002; Years reports issued: 2005, 2006[C,D]; Year press release issued: None issued. Sources: GAO, BJS. [A] Contacts between Police and the Public, Findings from the 1999 National Survey, February 2001. [B] Characteristics of Drivers Stopped by Police, 1999, March 2002. [C] Contacts between Police and the Public, Findings from the 2002 National Survey, April 2005. [D] Characteristics of Drivers Stopped by Police, 2002, June 2006. [E] Force or Threatened Force Used in Less than 1 Percent of All Police- Public Interactions, March 2001. [End of table] Quality Guidelines for Statistical Products: Over the last several years, various types of guidance have been developed to help federal agencies such as BJS ensure the integrity of statistical information. In 1992, in response to requests from Congress and others as to what constitutes an effective statistical agency, the National Research Council began issuing best-practice guidelines.[Footnote 7] According to the Committee on National Statistics,[Footnote 8] which authored the guidelines, the guidelines have been widely cited and used by Congress and federal agencies, and have shaped legislation and executive actions to establish and evaluate statistical agencies. These recommended guidelines, which BJS and other statistical agencies may choose to voluntarily follow, cover the review, approval, and dissemination processes of products issued by federal statistical agencies. In its guideline document, Principles and Practices for Federal Statistical Agencies, the National Research Council indicated, among other things, that statistical agencies should provide high-quality data, take a strong position of independence, be perceived to be free of political interference and policy advocacy, and strive for wide dissemination of their results.[Footnote 9] In particular, according to the National Research Council, the quality guidelines are to cover the: * review process and include verification of sources and results, disclosure of limitations, and accuracy of results; * approval process and include who has authority over the content and timing of the release of a product, and separation of policy from statistical information; and: * dissemination process and include the usability of information and its accessibility to a wide range of people. In February 2002 and September 2006, pursuant to the Information Quality Act of 2001, OMB issued policy and procedural guidance to federal agencies, including statistical agencies such as BJS, directing them to develop their own quality guidelines to help maximize the quality, objectivity, utility, and integrity of the information they disseminate.[Footnote 10] OMB stated that it was essential that federal statistics be collected, processed, and published in a manner that guarantees and inspires confidence in their reliability.[Footnote 11] Specifically, OMB directed federal agencies to "adopt a basic standard of quality — as a performance goal," and "take appropriate steps to incorporate information quality criteria into agency dissemination practices." In response to OMB's February 2002 guidance, the Department of Justice, Office of Justice Programs, and BJS issued their own guidelines later that year. BJS issued a second edition of its guidelines in 2005. In formulating its guidelines, BJS stated that it sought to provide the public with additional information regarding its methods for ensuring the quality, utility, objectivity, and integrity of the statistics it publicly disseminates. As a component of the Department of Justice, BJS is governed by its own data quality guidelines, as well as the information quality guidelines promulgated by the Office of Justice Programs, Department of Justice, and Office of Management and Budget. The Department of Justice's Information Quality Guidelines are intended to (1) provide the department's components with a foundation for developing their own, more detailed procedures, (2) provide guidance to component staff, and (3) inform the public of the agency's policies and procedures. The Office of Justice Programs' information quality guidelines require its components--including BJS--to (1) assess the usefulness of the information to be disseminated to the public by continuously monitoring information needs, developing new information sources, or revising existing methods, models, and information products where appropriate; (2) ensure disseminated information is accurate, clear, complete, reproducible, and presented in an unbiased manner by using reliable data sources, sound analytical techniques, and documenting methods and data sources; and (3) protect information from unauthorized access, corruption, or revision. As in the case of the Department of Justice's guidelines, the Office of Justice Programs provides guidance to its components in developing their own, more specific quality guidelines. BJS Followed Nearly All Available Quality Guidelines to Help Ensure Accuracy and Integrity of Products Issued from Police-Public Contact Surveys: BJS Fully Followed All Applicable Quality Guidelines for the Four Reports Issued from the 1999 and 2002 Police-Public Contact Surveys: For all four reports issued from the two Police-Public Contact Surveys, we found that BJS fully followed all of the review, approval, and dissemination guidelines available at the time of issuance. We considered a guideline to have been fully followed if our independent analysts determined that all aspects of the guideline were followed. (Our methodology for how we determined the extent to which BJS followed the guidelines is explained in app. I.) The extent to which BJS followed applicable, available guidelines when it issued its Police- Public Contact Survey reports is shown in table 2. Table 2: Guidelines That BJS Fully Followed for the Reports Issued from Its 1999 and 2002 Police-Public Contact Surveys: Year survey conducted: 1999; Year report issued: 2001; Number of applicable guidelines available when report was issued: 10; Number of applicable guidelines that BJS fully followed: 10. Year survey conducted: 1999; Year report issued: 2002; Number of applicable guidelines available when report was issued: 12; Number of applicable guidelines that BJS fully followed: 12. Year survey conducted: 2002[A]; Year report issued: 2005; Number of applicable guidelines available when report was issued: 23; Number of applicable guidelines that BJS fully followed: 23. Year survey conducted: 2002[A]; Year report issued: 2006; Number of applicable guidelines available when report was issued: 23; Number of applicable guidelines that BJS fully followed: 23. Source: GAO analysis: [A] The number of guidelines varied across the time periods covered. Across the time periods covered, there were a total of 24 guidelines applicable to the review, approval, and dissemination of Police-Public Contact Surveys products. In general, the number of guidelines increased between February 2001, when the first survey report was issued, and June 2006, when the latest survey report was issued. However, one National Research Council guideline was dropped when the council issued the second edition of its guidelines. Had this guideline not been dropped, 13 guidelines would have been applicable to BJS's 2002 survey report, instead of 12, and 24 guidelines would have been applicable to the 2005 and 2006 survey reports, instead of 23. Table 4 in appendix II reflects which guidelines were applicable to each of the survey products. [End of table] For the first report issued from the 1999 survey, we found that BJS voluntarily followed the National Research Council's 10 applicable existing guidelines; for the second report, we found that BJS voluntarily followed those 10, as well as 2 additional guidelines issued since the first report, for a total of 12. For each of the two reports based on the 2002 survey, we found that BJS followed all 23 available data quality guidelines that had by then been issued by the National Research Council, the Department of Justice, the Office of Justice Programs, and BJS itself.[Footnote 12] The data quality guidelines that BJS followed describe how agencies should review statistical information, obtain the approval of key decision makers, and publicly disseminate the information. While not all of the guideline-issuing organizations addressed the review, approval, and dissemination process, in total across the four organizations--the National Research Council, Department of Justice, Office of Justice Programs, and Bureau of Justice Statistics--all three areas were addressed. Some examples of the guidelines that BJS fully followed in its report issuance process are listed below. (For a complete list of all available data quality guidelines, see appendix II.) * Components of the Department of Justice and Office of Justice Programs will review all information dissemination products for their quality (including objectivity, utility, and integrity) before they are disseminated. * All BJS reports and other statistical products must be subject to an objective and appropriate verification process conducted by qualified BJS staff other than the author of the report. * The statistical agency has recognition by policy officials outside the statistical agency of its authority to release statistical information without prior clearance. * The statistical agency has authority for professional decisions over the scope, content, and frequency of data compiled, analyzed, or published. * The objectivity of BJS statistics must be vigilantly protected at all times by BJS staff. On the basis of our analysis, BJS successfully followed all applicable quality guidelines for these survey-based statistical reports, which both BJS and we consider to be statistical products covered by the guidelines. Thus, we believe the agency took proper steps to help ensure the accuracy and integrity of the review, approval, and dissemination processes associated with issuing public reports based on the two surveys we reviewed. BJS concurred with our analysis. All of the reports were posted to the BJS Web site, where the information is to be accessible to the general public.[Footnote 13] BJS Fully Followed 7 of 10 Applicable Data Quality Guidelines for the Single Press Release Issued from Its Police-Public Contact Surveys: For the single press release that was issued--that is, the 2001 press release based on BJS's 1999 Police-Public Contact Survey--we determined that BJS fully followed 7 of the 10 applicable National Research Council guidelines available at the time.[Footnote 14] The 7 federal data quality guidelines that BJS fully followed are listed below. (1) A statistical agency should develop an understanding of the validity and accuracy of its data and convey the resulting measures of quality to users in ways that are comprehensible to nonexperts. (2) A statistical agency should use modern statistical theory and sound statistical practice in all technical work. (3) A statistical agency has maintenance of a clear distinction between statistical information and policy interpretations of such information by the President, the secretary of the department, or others in the executive branch. (4) A statistical agency should follow good practice, in reports and other data releases, in documenting concepts, definitions, data collection methodologies, and measures of uncertainty, and in discussing possible sources of error. (5) Effective dissemination programs include policies for the preservation of data that guide what data to retain and how they are to be archived for future secondary analysis. (6) An agency should have an established publications policy that describes, for a data collection program, the types of reports and other data released to be made available, the audiences served, and the frequency of release. (7) Dissemination of data and information (basic series, analytic reports, press releases, public use tapes) should be timely and public. Avenues of dissemination should be chosen to reach as broad a public as reasonably possible. There were 3 other applicable quality guidelines that we determined BJS was not in a position to follow in connection with this issued press release, and a press release based on the 2002 Police-Public Contact Survey findings was not issued. It is important to note that, for reasons discussed later in this report, BJS officials did not believe these guidelines were applicable to its press releases in the first place. Differing Views on Applicability of Guidelines Raised Questions about Clarity, Differing Views on Guideline Applicability Raised Questions About Their Clarity, while the Roles of Certain Noncareer Appointees during During the Product Issuance Process Affected BJS's Independence: Two key factors affected whether and how BJS followed quality guidelines during the review, approval, and dissemination of products issued from the 1999 and 2002 Police-Public Contact Surveys. Two key factors had an impact on whether and how BJS voluntarily followed quality guidelines during the review, approval, and dissemination processes followed for the press releases that could be issued from the 1999 and 2002 Police-Public Contact Surveys. First, while BJS believed, as noted earlier, that its survey reports were statistical products covered by the quality guidelines, it did not believe that BJS believed certain quality guidelines did not apply to the survey-related press releases was a statistical product because it did not consider them to be statistical products covered by the quality guidelines, and . BJS cited a lack of specificity in the National Research Council's guidelines available at the time as a basis for this conclusion. We believe, however, that while BJS's interpretation of the guidelines was not unreasonable, there was nonetheless sufficient evidence to indicate for a different interpretation; namely, that this press release was a statistical product, that the available guidelines did apply, and that BJS was not in a position to meet three3 of 10 guidelines, in particular, for the single press release issued from the 1999 survey, owing to a second factor. This second factor was the role that certain noncareer appointees outside BJS have the ability to play, pursuant to Department of Justice policy, in the product issuance process. In certain instances, the roles of these non-BJS officials meant that BJS was not in a position to fully follow all guidelines related to agency independence, and this holds the potential for future actual or perceived political interference in BJS's product issuance process for statistical products. BJS Interpreted Guidelines as Not Applying to Press Releases, Raising Questions about the Applicability and Clarity of Certain Guidelines: In both written documentation and oral comments, BJS officials stated that they believed they were in full conformance with the National Research Council's guidelines and disagreed with our determination that the agency was not in a position to follow 3 of 10 guidelines for the 2001 Police-Public Contact Survey press release that was issued from the 1999 survey. The guidelines that we determined BJS was not in the position to fully follow all pertain to the agency's independence and, in particular, to its control over the issuance of press releases. These guidelines were: (1) The statistical agency has recognition by policy officials of its authority to release statistical information without prior clearance. (2) The statistical agency has authority for professional decisions over the scope, content, and frequency of data compiled, analyzed, or published. (3) The release of information should not be subject to actual or perceived political interference. In particular, the timing of the public release of data should be the responsibility of the statistical agency. BJS officials asserted that, based on their interpretation of the National Research Council's guidelines, BJS press releases did not qualify as statistical products and, therefore, press releases did not fall within the purview of the council's guidelines. They also asserted that neither BJS's own quality guidelines, nor those issued by the Department of Justice and the Office of Justice Programs, apply to BJS press releases. Both BJS and Office of Justice Programs officials stated that the applicability of the council's guidelines to BJS press releases was, at a minimum, open to question because the council did not state that press releases are data disseminations. In other words, according to BJS and the Office of Justice Programs, press releases are not publications of data, but rather they are simply announcements that a data publication is forthcoming. In its communications with us, BJS stated that many of the guidelines do not apply to press releases but apply only to statistical products. Based on its content rather than its label as a press release, and notwithstanding that the policies and procedures for developing and issuing products labeled by the Office of Justice Programs as press releases differed from policies and procedures for products it labeled as statistical products, we believe there is sufficient evidence for us to conclude that the press release issued that BJS jointly issued from the 1999 Police-Public Contact Survey qualified as a statistical product to which the National Research Council's quality guidelines appropriately apply. Our analysis of this press release indicated that it was a data-based statistical product, more than simply an announcement that a data publication was forthcoming. In its entirety, the press release consisted of 20 sentences and one table describing the survey's statistical findings; 3 sentences on the survey's methodology; and 5 sentences on who prepared the report and how to obtain copies. We found that this press release was a compilation of statistical data that contained no interpretations, conclusions, or policy statements.[Footnote 15] (See Appendix III for a reproduction of the press release.) In accordance with the council's guidelines, the release maintained "a clear distinction between statistical information and policy interpretations of such information." To understand whether the National Research Council was purposeful in not stating that its guidelines were applicable to statistical agency press releases, we contacted the council to seek clarification. Officials from the council's Committee on National Statistics, which authored the data quality guidelines, stated that although the Principles and Practices document did not specifically state that the guidelines covered the content, scope, and timing of press releases issued by statistical agencies, it was not the committee's intent to exclude press releases from the guidelines. They stated that, in their view, press releases issued by BJS are statistical products to which it is appropriate to apply the guidelines. BJS and we agree that the National Research Council's guidelines apply, in general, to statistical products. In asserting that the press release that BJS jointly issued with the Department of Justice and Office of Justice Programs was not a statistical product, BJS correctly noted that the National Research Council did not explicitly state that the guidelines covered press releases. However, given the strong statistical content of the Police-Public Contact Survey press release, we did not believe that such an explication was necessary. Nonetheless, we acknowledge that it is not unreasonable for BJS to could reach a different conclusion given the lack of specificity that existed in the council's printed guidelines. Because BJS's own data quality guidelines, issued in 2002, state that they "govern all justice statistics that BJS produces and disseminates for the general public, including all statistics that are featured in BJS publications, on the website, and in BJS press releases," we considered the BJS guidelines to be applicable to press releases, as well. BJS, however, did not hold this view. It is important to note that we are not finding fault with BJS for the conclusions it drew with respect to the applicability of the quality guidelines to its press release issuance process because the National Research Council's guidelines were not explicit on this matter. Indeed, we noted in our May 2006 report on data quality that 2 of 14 statistical agencies we surveyed stated that there was ambiguity as to whether a statistical press release was a statistical product, and if so, whether statistical agencies could issue them without first getting releases cleared at the departmental level. BJS was among the 14 statistical agencies surveyed, but it was not one of the two agencies reporting ambiguity in whether a statistical press release was a statistical product. Overall, we believe that BJS made a good faith effort to follow the guidelines it deemed to be applicable to the Police-Public Contact Survey products. Deciding which guidelines a statistical agency like BJS should follow is further complicated by the fact that BJS's parent organizations--the Department of Justice and Office of Justice Programs--have explicitly stated that their own guidelines do not apply to press releases. However, these organizations' guidelines are intended to be broadly applicable to both statistical and nonstatistical agencies. For example, the Department of Justice comprises 38 separate component organizations that produce a variety of types of information, both statistical and nonstatistical in nature. The Office of Justice Programs is composed of 6 bureaus and program offices, and these, too, produce both statistical and nonstatistical information. Because we believe that press releases issued by the department and the Office of Justice Programs may in some, but not all, instances be statistical products, we do not hold the view that statistical guidelines should be universally applicable to all press releases issued by the Department of Justice and Office of Justice Programs. However, because different interpretations can arise, we believe that clarification regarding which guidelines should be applied under which circumstances--and, specifically, to press releases--would be helpful to statistical agencies that are in situations similar to BJS's. To address potential discrepancies such as these, in a May 2006 report on the quality of federal data,[Footnote 16] we recommended that to help improve governmentwide data dissemination practices that would further safeguard the integrity of federal statistical data, OMB should consider how best to address the gaps we identified between agencies' data dissemination practices and the National Research Council's guidelines. We noted in that report that OMB, in concert with federal statistical agencies, was developing a governmentwide directive on the release and dissemination of statistical products that, according to OMB officials, parallels the council's and other generally accepted dissemination practices. We pointed out that it will be important for OMB's directive to consider, for example, whether the directive should cover principal statistical agencies only, the statistical functions of all agencies, or only statistical products. OMB officials indicated that the guidance is intended to help ensure that statistical products are policy-neutral, timely, and accurate. We recommended that, among other things, OMB include in this directive (1) clear definitions of what is and is not covered, (2) the extent to which agencies should document their data dissemination guidance and how often the guidance should be reviewed, (3) the amount of flexibility agencies have in implementing OMB's guidance, and (4) procedures for monitoring agencies' adherence to its directive. To the extent that statistical agencies appropriately follow these practices, the directive could promote more consistent adherence to practices that facilitate broader dissemination of statistical data and enhance its credibility. Although OMB did not provide comments on the recommendations in our 2006 report, an OMB official told us that as of January 2007, OMB was still working on this directive.[Footnote 17] We believe it remains important for OMB to complete its directive on the release and dissemination of statistical products in order to help safeguard the integrity of federal statistical data, reduce the likelihood that the type of disagreement discussed in this report would recur, and help assure both the actual and perceived independence of BJS. The Roles Played by Certain Outside Noncareer Appointees in the Press Release Process Affected BJS Carrying Out Its Role as an Independent Statistical Agency: The second key In addition to the role that lack of specificity played in BJSs actions with respect to following certain guidelines for its issued press release, we believe there is another Another factor that contributed to whether that affected whether and how had an impact on affected BJS followed guidelines was in a position to follow these guidelines in carrying out its role as an independent statistical agency concerned the involvement of noncareer appointees outside of BJS in the press release issuance process, and had implications for BJS's independence as a statistical agency. of noncareer appointees outside of BJS . Specifically, we determined that BJS was not in a position to fully follow the 3 National Research Council guidelines listed above in the previous section for the 2001 press release based on the 1999 survey (the only applicable, available data quality guidelines in place in 2001) because certain noncareer appointees outside of BJS and within the Department of Justice, are vested--pursuant to the Department of Justice's and Office of Justice Programs' policies defining the roles and responsibilities of their noncareer appointees--with the ability to participate in the review, approval, and dissemination of press releases.[Footnote 18] In certain cases, the roles and responsibilities of these noncareer appointees precluded BJS from being in the position to fully follow certain guidelines. Specifically, the Assistant Attorney General within the Department of Justice's Office of Justice Programs has general statutory responsibilities with respect to coordinating the activities of that office and its various components, such as BJS.[Footnote 19] These statutory provisions do not specifically address the Office of Justice Programs's role with respect to the review, approval, and dissemination of press releases. However, under departmental policy, noncareer appointees within the Department of Justice and outside of BJS have the ability to participate in the press release issuance process. Table 3 shows the type of involvement that the Assistant Attorney General in the Office of Justice Programs and other noncareer appointees generally have had in the press release review, approval, and dissemination process. Appendix IV describes in more detail the responsibilities of these various officials associated with review, approval, and dissemination procedures for both BJS reports and press releases. Table 3: Noncareer Appointees' Roles in the Review, Approval, and Dissemination of BJS Press Releases: Area of involvement: Review; Entity involved in press release process: Bureau of Justice Statistics: Director: X; Entity involved in press release process: Office of Justice Programs: Chief of Staff: X; Entity involved in press release process: Office of Justice Programs: Deputy Assistant Attorney General: X; Entity involved in press release process: Office of Justice Programs: Assistant Attorney General: X; Entity involved in press release process: Department of Justice: Office of Public Affairs Director: X. Area of involvement: Approval; Entity involved in press release process: Bureau of Justice Statistics: Director: X; Entity involved in press release process: Office of Justice Programs: Chief of Staff: X; Entity involved in press release process: Office of Justice Programs: Deputy Assistant Attorney General: X; Entity involved in press release process: Office of Justice Programs: Assistant Attorney General: X; Entity involved in press release process: Department of Justice: Office of Public Affairs Director: X. Area of involvement: Dissemination; Entity involved in press release process: Bureau of Justice Statistics: Director: X[A]; Entity involved in press release process: Office of Justice Programs: Chief of Staff: [Empty]; Entity involved in press release process: Office of Justice Programs: Deputy Assistant Attorney General: [Empty]; Entity involved in press release process: Office of Justice Programs: Assistant Attorney General: [Empty]; Entity involved in press release process: Department of Justice: Office of Public Affairs Director: X[B]. Source: BJS and Office of Justice Programs. [A] To the BJS web site, after approval by the BJS Director. [B] To Congress, the media, and executive department press offices, after verification by the BJS Director. [End of table] With respect to the first of the three guidelines, which calls for a statistical agency to have authority to release information without prior clearance, it is our view that BJS was not in a position to follow this independence-related guideline at all because it did not have the ability to do so. This is because press releases are subject to review and approval by not only the BJS Director, but also by other Department of Justice noncareer appointees.[Footnote 20] Outside of BJS, the noncareer appointees participating in the clearance process are located in the Department of Justice's Office of Justice Programs (these include the Office's Chief of Staff, Deputy Assistant Attorney General, and Assistant Attorney General) and Office of Public Affairs. The current Assistant Attorney General and two former Assistant Attorneys General in the Office of Justice Programs told us that there is no written, formal policy or guidance that bounds their input and decision-making roles and responsibilities with respect to BJS press releases. BJS and OJP officials indicated that the Office of Justice Programs' Assistant Attorney General has ultimate responsibility for the review and approval of BJS press releases. Press releases are issued jointly on letterhead listing BJS and the Department of Justice. The current BJS Director confirmed that publication and dissemination functions for press releases are considered to be within the Assistant Attorney General's oversight authority. Because the National Research Council stated that an aspect of independence includes "recognition by policy officials outside the statistical agency of its authority to release statistical information without prior clearance," we concluded that BJS was not in the position to follow this guideline because, as we have stated, we believe the Police-Public Contact Survey press release was a statistical product that BJS could not issue independently. In practice, the ways in which Assistant Attorneys General of the Office of Justice Programs have exercised their authority have varied. For instance, one former Office of Justice Programs' Assistant Attorney General told us that she placed "self-imposed" limits on her decisions to modify the content of a BJS press release based on her awareness of congressional support for, and her own belief in, the independence of statistical agencies. The current Office of Justice Programs' Assistant Attorney General told us that she reviews only press releases that contain quotes from the Attorney General. She said that since she assumed her position in 2005, there have been no BJS press releases that have quoted the Attorney General, and she has relied on her Deputy Assistant Attorney General, the BJS Director, and others to ensure the accuracy and clarity of press releases. Nevertheless, the BJS Director must obtain the approval of the Office of Justice Programs' Assistant Attorney General and other Justice noncareer appointees to issue a press release. With respect to the second guideline, pertaining to the agency's decisions over the scope, content, and frequency of data compiled, analyzed, or published, we found that BJS was not in a position to fully follow this independence-related guideline. Specifically, we found that BJS could exercise professional decisions about the frequency of data analyzed and published (within available budgets), but did not always have complete control over the scope and content of survey press releases to be issued. As noted above, this was due to the fact that press releases are joint products of BJS, the Office of Justice Programs, and the Department of Justice, and noncareer appointees outside of BJS can become involved in the press release process. BJS's situation with respect to this second guideline came to the fore during the drafting of a press release in 2005 based on the 2002 Police-Public Contact Survey. The press release that BJS sought to publish would have included the following statistical findings from the accompanying Police-Public Contact Survey report: (1) there was no statistically significant difference between the rates that white and minority drivers reported being stopped by police, and (2) once stopped, a larger percentage of black and Hispanic minority drivers reported police using or threatening to use force against them than did whites. The then-BJS Director and the then-Acting Assistant Attorney General had a difference of opinion regarding the presentation of the second statistical finding, which was included in the Police-Public Contact Survey report. Despite reported efforts on the part of both parties to negotiate alternative language with respect to the content of the press release, they could not resolve their differences and the BJS Director decided that a press release would not be issued. The current BJS Director told us that it is "inconceivable" that the Assistant Attorney General would issue a press release without the BJS Director's prior approval. According to current BJS officials (both career and noncareer) and the Office of Justice Programs' Office of Communications staff, during the period 1996-2006, this was the only instance in which a BJS press release was prepared but not issued because the Office of Justice Programs and BJS could not agree on the contents. In all other instances during this period, according to these officials, when the parties disagreed on the content of a press release, they were able to resolve their differences. With respect to the third guideline, pertaining to actual or perceived political interference and the timing of a release, we similarly believe BJS was in not in a position to fully follow this independence- related guideline for the 2001 press release, which, as discussed earlier, we believe to be a statistical product. Although we found no evidence of political interference with the timing of the 2001 survey press release issued from the 1999 survey, we found that BJS does not have complete control over the timing of press releases, as recommended by the National Research Council. Since both noncareer appointees and career officials in the Office of Justice Programs and the Department of Justice have a role in reviewing and approving BJS press releases, they can affect the date that a press release is issued. According to BJS, career and noncareer appointees outside of BJS can delay the issuance of a press release for reasons having nothing to do with political interference, such as a determination that the press release is not sufficiently newsworthy at the time that it was designated to be issued. On balance, we believe that the noncareer appointees who played decision-making roles in the Police-Public Contact Survey press release process that we reviewed acted within the scope of the roles and responsibilities accorded them under Department of Justice policies, and that BJS made a reasonable effort to adhere to all applicable data quality guidelines. The fact that certain noncareer officials have the ability to make decisions that affect BJS's ability to fully meet federal data quality guidelines suggests, however, that the potential exists for BJS's review, approval, and dissemination process for statistical products to be subject to political interference. Thus, certain actions by noncareer appointees--though made on the basis of professional judgment--could put them at odds with the very guidelines designed to ensure the statistical independence and integrity of agencies such as BJS. Agency Comments and Our Evaluation: We provided a draft of this report to the Department of Justice for review and comment. On March 13, we received written comments on the draft report from the Office of Justice Programs' Assistant Attorney General, and the comments are reproduced in full in appendix VI. In her letter, the Assistant Attorney General affirmed several of our findings and agreed that a need exists for clear definitions about what quality guidelines cover. She noted that competing interpretations exist about what constitutes a statistical product and that the federal statistical community would benefit from clarity in this area. However, the Assistant Attorney General disagreed with our characterization of the 2001 Police-Public Contact Survey press release as a statistical product and, therefore, with our conclusion that the National Research Council's quality guidelines applied to this press release. The Assistant Attorney General stated that "a press release — is a public relations announcement issued to encourage media coverage. The mere presence of statistics in a press release does not transform a press release into a statistical product." We do not believe and have not stated that the mere presence of statistics in a press release in and of itself transforms it into a statistical product any more than we believe or have stated that labeling a document lacking in statistics but called a statistical product necessarily transforms it into one. The Assistant Attorney General also stated that we "mischaracterized" BJS's data quality guidelines as applying to press releases because the guidelines apply only to the statistics contained in BJS press releases, and because BJS conforms with OMB, the Department of Justice, and the Office of Justice Programs in considering press releases to be outside the scope of the guidelines. For the following reasons, we maintain that we made a sound decision in applying BJS's guidelines to the Police-Public Contact Survey press release: (1) BJS's guidelines state that they "govern all justice statistics that BJS produces and disseminates for the general public, including all statistics that are featured in BJS publications, on the website, and in BJS press releases;" and (2) the Police-Public Contact Survey press release was made up almost entirely of survey statistics, indicating to us that it was a statistical product. In determining that the Police-Public Contact Survey was a statistical product, we felt that the content of the press release was a more important determinant than the label attached to it, or the fact that the processes and staff involved in developing the press release were different from those in BJS reports. The Assistant Attorney General also noted that the National Research Council's written guidelines did not explicitly cover press releases. Because we agree, we contacted the National Research Council and consulted with officials of the Council's Committee on National Statistics (the authoring committee of the Principles and Practices). The officials concurred with our view that BJS press releases referring to statistical products (as opposed to press releases about the announcement of a new agency head, for example) are statistical products to which it is appropriate to apply the guidelines. Although the Principles and Practices document did not specifically state that the guidelines covered the content, scope, and timing of press releases issued by statistical agencies, according to these officials it was not the Committee's intent to exclude such press releases from the guidelines. The Assistant Attorney General also felt that the draft report overstated the potential threats to BJS's independence because we used the term "statistical products" to refer to press releases. She was concerned with our observation that the potential exists for BJS's review, approval, and dissemination process for statistical products to be subject to political interference since noncareer officials can affect BJS's ability to meet federal data quality guidelines. We stand by this conclusion. Department of Justice policy permits noncareer appointees within the Department but outside of BJS to participate in the press release process. At the same time however, we are unaware of anything that prevents future modifications to that policy to similarly allow noncareer appointees to participate in BJS's report issuance. Thus, we believe that we have correctly assessed the risk of potential or actual threats to BJS's independence. Finally, the Assistant Attorney General stated that even if the council's written guidelines explicitly applied to press releases, the BJS director would not adhere to them and no current law can make him do so. We recognize that they are voluntary and not legally required and never have said otherwise. As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies of the report to the Attorney General, the Director of the Office of Management and Budget, and other interested parties. In addition, the report will be available at no charge on GAO's home page at http://www.gao.gov. Please contact Brian Lepore at (202) 512-4523 or leporeb@gao.gov if you or your staff have any questions concerning this report. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix VII. Signed by: Brian Lepore, Acting Director: Homeland Security and Justice Issues: List of Congressional Addressees: The Honorable John Conyers: Chairman: Committee on the Judiciary: House of Representatives: The Honorable William Delahunt: The Honorable Sheila Jackson Lee: The Honorable Zoe Lofgren: The Honorable Martin Meehan: The Honorable Jerrold Nadler: The Honorable Linda Sanchez: The Honorable Debbie Wasserman Schultz: The Honorable Robert C. "Bobby" Scott: The Honorable Mel Watt: The Honorable Robert Wexler: House of Representatives: [End of section] Appendix I: Objectives, Scope, and Methodology: This report addresses the following two objectives for the 1999 and 2002 Police-Public Contact Surveys, the two surveys for which products had been issued as of February 2007: (1) To what extent did the Bureau of Justice Statistics (BJS) follow available guidelines to help ensure the accuracy and integrity of the review, approval, and dissemination of reports and press releases based on its surveys? (2) What key factors affected whether and how BJS followed available guidelines? In addition, we provide information on scope and methodology changes in the Police-Public Contact Surveys over time (see app. V). To address the first objective, regarding the extent to which BJS followed guidelines, we obtained quality guideline documents from BJS, the Department of Justice's Office of Justice Programs, the Department of Justice, and the National Research Council. The guidelines that we obtained from these organizations covered the period between February 2001, when the first product based on the 1999 Police-Public Contact Survey was issued, and June 2006, when the most recent product based on the 2002 survey was issued. We included these federal organizations in our review because BJS is a component of the Office of Justice Programs, which in turn is a component of the Department of Justice, and BJS considers itself to be "governed by" the information quality guidelines of these organizations. We included the National Research Council in our review because it is a widely recognized organization that issued guidelines that were intended to be statements of best practice and provide information on what constitutes an effective statistical organization. We also reviewed guidance and directives issued by the Office of Management and Budget (OMB) because OMB is charged with issuing governmentwide policy and procedural guidance to federal agencies, which are then encouraged to issue their own implementation guidelines. We took several steps to determine the extent to which BJS followed the specific quality guidelines that it, the Office of Justice Programs, the Department of Justice, and the National Research Council had issued. From the documents provided by these four organizations, a GAO analyst initially identified a total of 63 guidelines that pertained to product review, approval, and dissemination processes. For verification purposes, a GAO methodologist also reviewed the guideline documents. The GAO methodologist agreed with the auditor that all 63 guidelines were appropriate for inclusion in our review. Because many of the guidelines issued by the four organizations were similar and overlapping, the GAO auditor reduced the list to 24 nonduplicative guidelines. The GAO methodologist again reviewed the work of the auditor, and in all cases agreed with the auditor that similar guidelines were being appropriately grouped. We then developed a data collection instrument to determine whether BJS was following guidelines for the 1999 and 2002 Police-Public Contact Surveys, on which information could be recorded as to whether BJS fully followed, partially followed, or did not at all follow each of the guidelines. We defined "fully" as all aspects of the guideline being followed; "partially" as some, but not all, aspects of the guideline being followed; and "not at all" as no aspects of the guideline being followed. We asked BJS to complete a separate data collection instrument with respect to each of its 1999 and 2002 Police-Public Contact Survey reports and one press release, and to support each response by providing documentary evidence. To decrease the burden on BJS, GAO analysts completed the data collection instrument for 9 of the 24 guidelines, for which we already had sufficient information (for example, documents describing agency processes and procedures, and interviews regarding the roles and responsibilities of noncareer appointees).[Footnote 21] We provided our assessments regarding these guidelines to BJS and asked officials to either confirm or not confirm them. Two GAO analysts reviewed BJS's responses and all available supporting documentary and testimonial evidence, and determined whether BJS fully, partially, or did not at all meet each guideline. We provided our findings to BJS for review and comment. BJS's 1996 and 2005 Police-Public Contact Surveys were outside the scope of our work. We excluded the 1996 survey because that was a relatively small-scale pilot study; and we excluded the 2005 survey, the most recent Police-Public Contact Survey conducted, because no reports or press releases have yet been issued from this work. To address the second objective, regarding key factors that affected whether and how BJS followed guidelines, we reviewed processes and procedures that described the review, approval, and dissemination processes for BJS-generated reports and press releases, with particular interest in identifying the roles of noncareer appointees involved in each of these processes. We also reviewed pertinent statutory provisions relating to the roles and responsibilities of officials with respect to BJS. We conducted in-person interviews with, or obtained written responses to our questions from, noncareer appointees in BJS, the Office of Justice Programs, and the Department of Justice's Office of Public Affairs. Specifically, we conducted in-person interviews with the current BJS Director and the BJS Director who was involved in the disagreement with the Acting Assistant Attorney General, as well as with the current Assistant Attorney General and Deputy Assistant Attorney General in the Office of Justice Programs. We obtained detailed written responses to our questions from a former BJS Director, the Acting Assistant Attorney General who was involved in the disagreement with the BJS Director, and two former Assistant Attorneys General from the Office of Justice Programs. We conducted a telephone interview with the current Deputy Director of the Department of Justice's Office of Public Affairs. Among other things, we asked these noncareer appointees to provide us with information about BJS's process for reviewing, approving, and disseminating reports and press releases; the roles and responsibilities of noncareer appointees in that process; changes, if any, that had occurred in the roles played by noncareer appointees; procedures used to help ensure that BJS reports and press releases were accurate, reliable, and unbiased; and any factors that may have affected BJS's independence in the product issuance process. Finally, we reviewed the guidelines of BJS, the Department of Justice's Office of Justice Programs, the Department of Justice, and the National Research Council to determine that they reflected the product issuance processes and to consolidate them in order to eliminate duplication. To determine what changes, if any, have occurred in the scope and methodology of the Police-Public Contact Surveys between 1996 and 2006, which we present in appendix V, we initially developed a matrix of key scope and methodology dimensions, based on a review of the standard social science literature. We then conducted interviews and reviewed documents with respect to these dimensions, for all four Police-Public Contact Surveys--the 1996 pilot survey and the surveys of 1999, 2002, and 2005. We interviewed the current BJS Director and a former BJS Director, and available report authors and the key statistician participating in administrations of the survey, to ascertain their views concerning the intended scope of the four surveys, the methodologies used, scope and methodology changes that were made, and reasons for any changes. We also obtained written responses to our questions from these officials. We conducted a detailed documentary review of the scoping and methodology sections of the issued Police- Public Contact Survey reports and press releases, and extracted information about changes in the data collection instruments used (for example, the numbers and types of questions asked about searches and the use of force). In addition, we reviewed documents prepared by the American Statistical Association and the U.S. Bureau of the Census, which conducted field tests to ensure that Police-Public Contact Survey questions were appropriately devised. In cases where we noted that changes had been made between surveys, we reviewed Census Bureau documentation and interviewed staff and officials at BJS. It was beyond the scope of this review to address any personnel issues that may have arisen in connection with the disagreement over the content of the 2005 draft press release based on the 2002 Police-Public Contact Survey. We conducted our work between April 2006 and January 2007 in accordance with generally accepted government auditing standards. [End of section] Appendix II: Applicable Quality Guidelines for Statistical Products Issued by the Bureau of Justice Statistics and Other Agencies: BJS followed numerous recommended data quality guidelines designed to help ensure the accuracy and integrity of the Police-Public Contact Survey products that it issued in 2001, 2002, 2005, and 2006 based on its 1999 and 2002 surveys. The product issuance guidelines were used to aid BJS's efforts to review, approve, and disseminate these statistical products to the public and others. The guidelines were issued at various points in time by the following organizations: * the National Research Council, * the Bureau of Justice Statistics, * the Department of Justice, and: * the Office of Justice Programs. In addition to reviewing the guidelines of these four organizations, we also reviewed guidelines and directives issued by the Office of Management and Budget (OMB). However, we did not specifically assess BJS's practice with respect to following OMB's guidelines because OMB issued governmentwide policy and procedural guidance to federal agencies that called for agencies to develop their own implementing guidelines. Table 4 shows the guidelines that were available at the time BJS's 1999 and 2002 Police-Public Contact Survey products were issued, and which guidelines BJS was in a position to follow. Table 4: Extent to Which BJS Followed Available Quality Guidelines for 1999 and 2002 Police-Public Survey Products: Guideline: A statistical agency should follow good practice, in reports and other data releases, in documenting concepts, definitions, data collection methodologies, and measures of uncertainty, and in discussing possible sources of error; Issuing entity: NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: An effective dissemination program includes policies for the preservation of data that guide what data to retain and how they are to be archived for future secondary analysis; Issuing entity: NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: An agency should have an established publications policy that describes, for a data collection program, the types of reports and other data released to be made available, the audience served, and the frequency of release; Issuing entity: NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: Dissemination of data and information (basic series, analytic reports, press releases, public-use tapes) should be timely and public. Avenues of dissemination should be chosen to reach as broad a public as reasonably possible; Issuing entity: NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001- 2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was unavailable during the product issuance stage. Guideline: Statistical agency has dissemination policies that foster regular, frequent release of major findings from an agency's statistical programs to the public via the media, the Internet, and other means; Issuing entity: NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005- 2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: The statistical agency has maintenance of a clear distinction between statistical information and policy interpretations of such information by the President, the secretary of the department, or others in the executive branch; Issuing entity: NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: The statistical agency has authority for professional decisions over the scope, content, and frequency of data compiled, analyzed, or published. Most statistical agencies have broad authority, limited by budgetary constraints, departmental requirements, OMB review, and congressional mandates; Issuing entity: NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: BJS was not in the position to follow this guideline fully; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: The statistical agency has recognition by policy officials outside the statistical agency of its authority to release statistical information without prior clearance; Issuing entity: NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: BJS was not in the position to follow this guideline at all; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: Release of information should not be subject to actual or perceived political interference. In particular, the timing of the public release of data should be the responsibility of the statistical agency; Issuing entity: NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: BJS was not in the position to follow this guideline fully; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005- 2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: The objectivity of BJS statistics must be vigilantly protected at all times by BJS staff; Issuing entity: BJS; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: All BJS reports and other statistical products must be subject to an objective and appropriate verification process conducted by qualified BJS staff other than the author of the report; Issuing entity: BJS; Products issued from 1999 survey: Products issued in 2001- 2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: BJS staff must archive all analysis and data sets as soon as they become final in order to ensure the reproducibility of the published statistics; Issuing entity: BJS; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: BJS statistical products must provide sufficient information concerning instances where data have been imputed; Issuing entity: BJS; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: Effective dissemination program includes a variety of avenues for data dissemination, chosen to reach as broad a public as reasonably possible. Channels of dissemination include, but are not limited to, an agency's Internet Web site, government depository libraries, conference exhibits and programs, newsletters and journals, e-mail address lists, and the media for regular communication of major findings; Issuing entity: BJS, NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005- 2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: A statistical agency should develop an understanding of the validity and accuracy of its data and convey the resulting measures of quality to users in ways that are comprehensible to nonexperts; Issuing entity: NRC, BJS; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005- 2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: An effective dissemination program includes release of data in a variety of formats so that the information can be accessed by users with varying skills and needs for data retrieval and analysis; Issuing entity: BJS, NRC; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005- 2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: A statistical agency should use modern statistical theory and sound statistical practice in all technical work; Issuing entity: DOJ, NRC; Products issued from 1999 survey: Products issued in 2001- 2002: Press release[A]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was available and BJS followed it fully; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: OJP/DOJ will allow adequate time for reviews, consistent with the level of standards required for the type of information to be disseminated; Issuing entity: DOJ, OJP; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: OJP/DOJ will ensure compliance with relevant quality guidelines (i.e., utility, objectivity, and integrity requirements); Issuing entity: DOJ, OJP; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005- 2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: OJP/DOJ will provide methodologies, origins of data, limitations of the information, etc., whenever possible, as part of information dissemination; Issuing entity: DOJ, OJP, BJS; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: OJP/DOJ components will ensure disseminated information, as a matter of substance and presentation is accurate, reliable and unbiased. Objectivity is achieved by using reliable data sources, sound analytical techniques, and documenting methods and data sources; Issuing entity: DOJ, OJP; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005- 2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: OJP/DOJ components will review all information dissemination products for their quality (including objectivity, utility, and integrity) before they are disseminated; Issuing entity: DOJ, OJP; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: OJP/DOJ components will assess the usefulness of the information to be disseminated to the public; Issuing entity: DOJ, OJP; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005-2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Guideline: OJP/DOJ will ensure that the information fulfills the intentions stated and that the conclusions are consistent with the evidence; Issuing entity: DOJ, OJP; Products issued from 1999 survey: Products issued in 2001-2002: Press release[A]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report A[B]: Guideline was unavailable during the product issuance stage; Products issued from 1999 survey: Products issued in 2001-2002: Report B[C]: Guideline was unavailable during the product issuance stage; Products issued from 2002 survey: Products issued in 2005- 2006: Report C[D]: Guideline was available and BJS followed it fully; Products issued from 2002 survey: Products issued in 2005-2006: Report D[E]: Guideline was available and BJS followed it fully. Source: GAO analysis of DOJ, OJP, and BJS data. NRC: National Research Council: DOJ: Department of Justice: OJP Office of Justice Programs: [A] Force or Threatened Force Used in Less Than 1 Percent of All Police- Public Interactions (March 11, 2001). [B] Contacts between Police and the Public, Findings from the 1999 National Survey (February 2001). [C] Characteristics of Drivers Stopped by Police, 1999 (March 2002). [D] Contacts between Police and the Public, Findings from the 2002 National Survey (April 2005). [E] Characteristics of Drivers Stopped by Police, 2002 (June 2006). [End of table] [End of section] Appendix III: Press Release Issued in 2001 Based on BJS's 1999 Police- Public Contact Survey: [See PDF for image] [End of figure] [End of section] Appendix IV: BJS Processes for the Review, Approval, and Dissemination of Police-Public Contact Survey Products: Review, Approval, and Dissemination Process for Reports Issued from Police-Public Contact Surveys: Since the inception of the Police-Public Contact Survey in 1996, the BJS Director has been the single noncareer appointee who has had a decision-making role in BJS's review, approval, and dissemination processes for reports. The BJS Director is a noncareer presidential appointee subject to Senate confirmation. Figure 2 provides an overview of the process followed by BJS in the review, approval, and dissemination of Police-Public Contact Survey reports. Figure 2: Review, Approval, and Dissemination Process for BJS Survey Reports: [See PDF for image] Source: BJS. [End of figure] As indicated by the figure, the BJS report author and supervisor prepare the draft report for review and approval. The BJS Director reviews the draft, requests any changes, approves the final draft, and transmits a memorandum of notification through the Office of Justice Program's Assistant Attorney General up the chain of command to the Attorney General. The memorandum contains an abstract of the report, selected survey findings, and a projected release date for the report. BJS sets the release date for 30 days from the date that the Assistant Attorney General signs the memorandum of notification. The report is posted to the Web site at that time, or sooner if the date and time are specified in the notification memo. Review, Approval, and Dissemination Process for Press Releases Issued from Police-Public Contact Surveys: In contrast to the process followed for survey reports, several noncareer appointees are involved in the organizational review, approval, and dissemination process, as shown in figure 3. Figure 3: Review, Approval, and Dissemination Process for BJS Survey Press Releases: [See PDF for image] Sources: BJS and Office of Justice Programs. [End of figure] As indicated in the figure, in addition to the Director of BJS, there are three noncareer appointees within the Office of Justice Programs who participate in the review and approval process--the Chief of Staff, the Deputy Assistant Attorney General, and the Assistant Attorney General, and at least one noncareer appointee within Department of Justice headquarters: the Director of the Office of Public Affairs. The BJS report author and supervisor jointly work with staff from the Office of Justice Program's Office of Communications to prepare the press release. The BJS director reviews the draft press release, requests any changes, approves the final draft, and transmits the press release up the chain of command to the Office of Justice Programs' Assistant Attorney General for review and approval. Following approval by the Assistant Attorney General, the Department of Justice's Office of Public Affairs reviews the press release for clarity, and the BJS Director then verifies that the information in the press release is accurate. The Department of Justice's Office of Public Affairs is then responsible for disseminating the press release to Congress, the media, and executive department press offices, while BJS is responsible for disseminating the press release through its Web site. [End of section] Appendix V: Changes in the Scope and Methodology of the Police-Public Contact Surveys: BJS has conducted four Police-Public Contact Surveys as supplements to the National Crime Victimization Survey. The first Police-Public Contact Survey was conducted as a pilot in 1996. Three subsequent, more extensive surveys were conducted at 3-year intervals: 1999, 2002, and 2005. Although we do not discuss the 2005 survey in this report because no reports or press releases have yet been issued from this survey, we present information on the 2005 survey in this appendix because information is available on this survey's scope and methodology. The scope of the Police-Public Contact Surveys has consistently expanded over time, while the methodology has remained generally consistent. The Scope of the Police-Public Contact Survey Has Expanded with Each Additional Survey: The pilot survey was designed to test whether the survey could be effectively used as a supplement to the National Crime Victimization Survey to collect data on (a) the types of contacts the public have with the police, and (b) police use of force. To conduct this test, BJS employed a representative sample of 6,421 U.S. residents. The pilot survey yielded useful information on the various types of contacts the public had with the police, and whether force was used by the police. However, the sample size of the pilot survey was not sufficiently large for BJS to draw inferences about the extent to which the population at large would report that they experienced "excessive" use of force by police. For its Police-Public Contact Survey in 1999, BJS increased the sample size to a representative sample of 80,543 U.S. residents. The scope of the survey was further enhanced by adding questions about traffic stops (the most common form of public-police contact, as determined in the 1996 pilot survey), and including a question on whether the police used excessive force during any contact with the public. BJS officials told us that they added the traffic stop questions, at least in part, to "address the growing public concern about racial profiling in connection with traffic stops." In its 2002 survey, BJS expanded and refined its survey questions further. Specifically, according to BJS officials, they added questions that would help BJS estimate the extent to which U.S. residents nationwide would say that (1) they were stopped by the police while driving, (2) they or their vehicle were searched by the police without their permission during a traffic stop, and (3) they were arrested as a result of the search. In addition, BJS officials said that they added questions to estimate differences, if any, among racial groups in their rates of traffic stops at various times of the day, and whether police used force in situations where persons were engaged in such behaviors as arguing with, cursing, or disobeying police. In 2005, according to BJS officials, the scope of the Police-Public Contact Survey was further extended in several ways, including the following: (a) respondents were asked whether they had been arrested for driving under the influence of alcohol during the year (in order to make comparisons with Federal Bureau of Investigation (FBI) arrest rates, so that potential undercounting rates could be determined); (b) respondents were permitted to group themselves into any combination of racial categories (rather than choosing a single category) to better refine respondent demographic status; (c) respondents were asked whether police used force during any of their police contacts during the year, as opposed to the more limiting question in 2002, which was directed only toward the most recent contact with police; and (d) respondents were provided open-ended response fields on the survey instrument to indicate any ways they believed that the police had acted inappropriately toward them. The Police-Public Contact Survey Methodology Has Remained Consistent Following the 1996 Survey: The 1999, 2002, and 2005 Police-Public Contact Surveys have consistently maintained a similar methodology. The methodological dimensions of the surveys that have remained consistent are geographic coverage, target population, sampling design, data collection method, sample size/response rate, survey administration, and sample characteristics (as indicated in table 5). To illustrate, all three surveys involved selecting nationally representative, stratified, multi- cluster samples from the population of U.S. residents 16 years of age or older.[Footnote 22] The data were collected either through face- to- face or computer-assisted telephone interviews. The surveys were administered during the last 6 months of the year, and the demographic characteristics of the samples were similar across time periods. The 1996 pilot survey differed in several ways from the subsequent three surveys. Specifically, the pilot survey included residents younger than 16, included far fewer people than the subsequent surveys, and limited the sampling to individuals who had participated in the last round of the National Crime Victimization Survey. In addition, the percentage of face-to-face interviews was lower, and the survey administration period was shorter, and during a different time of the year, than in the other three surveys. Table 3: Police-Public Contact Survey (PPCS) Methodologies for the Four Surveys: 1996, 1999, 2002, and 2005: Dimensions: Geographic Coverage; 1996 survey[A]: National survey; 1999 survey: National survey; 2002 survey: National survey; 2005 survey: National survey. Dimensions: Target populations; 1996 survey[A]: Included: residents ages >12, living in the United States, including persons living in group quarters (e.g., dormitories, rooming houses, religious group dwellings); Excluded: crew members of merchant vessels, armed forces personnel living in military barracks, and institutionalized persons (e.g., correctional facility inmates); 1999 survey: Same as 1996, except the target population age range is >16; 2002 survey: Same as 1996, except the target population age range is >16; 2005 survey: Same as 1996, except the target population age range is >16. Dimensions: Sampling design; 1996 survey[A]: Nationally representative, stratified, multi-stage cluster sample, limited to respondents participating in the outgoing rotation panels of the National Crime Victimization Survey (i.e., the seventh and last round of interviews); 1999 survey: Nationally representative, stratified, multi-stage cluster sample, not limited to the out-rotation panels of the National Crime Victimization Survey; 2002 survey: Nationally representative, stratified, multi-stage cluster sample, not limited to the out-rotation panels of the National Crime Victimization Survey; 2005 survey: Nationally representative, stratified, multi-stage cluster sample, not limited to the out-rotation panels of the National Crime Victimization Survey. Dimensions: Data collection method; 1996 survey[A]: All survey interviews were collected using either the CATI (computer-assisted telephone interview) or PAPI (paper-and-pencil interview either in person or by telephone). Interviews: In person: 19% Telephone: 81%; 1999 survey: All PPCS interviews were collected using either the CATI or PAPI. Interviews: In person: 31% Telephone: 69%; 2002 survey: All PPCS interviews were collected using either the CATI or PAPI. Interviews: In-person: 34% Telephone: 66%; 2005 survey: All PPCS interviews were collected using either the CATI or PAPI. Interviews: In-person: 37% Telephone: 63%. Dimensions: Sample size/response rate; 1996 survey[A]: 6,421 (99%); 1999 survey: 80,543 (85%); 2002 survey: 76,910 (82%); 2005 survey: 63,943 (80%). Dimensions: Survey administration period; 1996 survey[A]: 5/1/96 to 7/ 31/96; 1999 survey: 7/1/99 to 12/31/99; 2002 survey: 7/1/02 to 12/31/ 02; 2005 survey: 7/1/05 to 12/31/05. Dimensions: Sample characteristics (unweighted percent); 1996 survey[A]: Gender: Male: 50.7 Female: 49.3. Race/ethnicity: White: 83.0 Black: 7.4 Hispanic: 5.7 Other: 3.9. Age: 16-19: 11.1 20-39: 44.5 40-59: 32.8 > 60: 11.6; 1999 survey: Gender: Male: 50.1 Female: 49.9. Race/ethnicity: White: 79.3 Black: 9.1 Hispanic: 8.4 Other: 3.2. Age: 16-19: 8.1 20-39: 45.6 40-59: 35.3 > 60: 11.1; 2002 survey: Gender: Male: 50.2 Female: 49.8. Race/ethnicity: White: 77.5 Black: 9.5 Hispanic: 9.8 Other: 3.2. Age: 16-19: 7.5 20-39: 44.6 40-59: 36.3 > 60: 11.7; 2005 survey: Gender: Male: 50.4 Female: 49.6. Race/ethnicity: White: 76.5 Black: 9.1 Hispanic: 9.6 Other: 3.9. Age: 16-19: 7.6 20-39: 41.3 40-59: 38.5 > 60: 12.6. Source: BJS data. Note: Percentages may not sum to 100 percent because of rounding. [A] Pilot survey. [End of table] [End of section] Appendix VI: Comments from the Department of Justice: U.S. Department of Justice: Office of Justice Programs: Office of the Assistant Attorney General: Washington, D.C. 20531: Mr. Brian Lepore: Acting Director: Homeland Security and Justice Team: Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Mar 12 2007: Dear Mr. Lepore: Thank you for the opportunity to review and comment on the Government Accountability Office (GAO) draft report entitled "Bureau of Justice Statistics, Quality Guidelines Generally Followed for Police-Public Contact Surveys, but Opportunities Exist to Help Assure Agency Independence" (GAO-07-340) (hereinafter "draft report"). The Office of Justice Programs (OJP) and its component, the Bureau of Justice Statistics (BJS), share GAO's concern that the development and dissemination of Federal statistics must be objective and of high integrity. The leadership of OR is committed to ensuring that the justice statistics produced by BJS adhere to the highest standards of quality and remain free from any political interference. We affirm the findings in the draft report that: * GAO found no evidence of any political interference with the timing of the 2001 survey press release or any other BJS document that GAO examined. * BJS successfully followed all applicable data quality guidelines, as determined by GAO to be relevant and appropriate, for the Police-Public Contact Survey (PPCS) statistical reports. BJS took proper steps to ensure the accuracy and integrity of the review, approval, and dissemination processes associated with issuing public reports based on the PPCS surveys reviewed by GAO. The BJS Director is the sole non- career appointee who has a decision-making role in these processes for report issuance. * Press releases issued in association with published reports are joint releases of BJS, OJP, and the U.S. Department of Justice (DOJ), and non- career appointees outside of BJS can become involved in the press release process. The non-career appointees who played decision-making roles in the PPCS press release process that GAO reviewed acted within the scope of the roles and responsibilities accorded them under DOJ policies. GAO based its assessment on the guidelines as presented by the National Research Council's (NRC's) Principles and Practices for a Federal Statistical Agency and the data quality guidelines of the DOJ, OJP, and BJS based on Office of Management and Budget (OMB) requirements. The NRC's Principles and Practices is a valuable and influential document for BJS and other Federal statistical agencies, and BJS has long accepted these principles as guides for effective practice. However, it is important to note that - by their own terms - the NRC guidelines are explicitly not designed to be prescriptions, and as such are followed by agencies solely in the exercise of their own discretion. It is equally important to note that NRC expressly stated in its guidelines that NRC does not expect every guideline to be followed.[Footnote 23] In sharp contrast, the information quality guidelines as promulgated by the OMB are mandatory and implement Pub. L. No. 106-554, providing guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information disseminated by Federal agencies. BJS is governed by the guidelines promulgated by OMB, DOJ, OJP, and BJS; adherence to these guidelines is not discretionary. GAO's draft report confirms OJP's declaration that, at all times, BJS followed all guidelines promulgated by these executive branch agencies. The findings in the draft report are predicated on GAO's assumption that a press release is a statistical product. Based upon this assumption, it is GAO's position that the NRC guidelines available in 2001 applied to press releases. We respectfully disagree with GAO's assumption. A press release simply is not a statistical product and thus should not be treated as a statistical product at all - let alone one that is somehow covered by the NRC guidelines. A press release, rather, is a public relations announcement issued to encourage media coverage. The mere presence of statistics in a press release does not transform a press release into a statistical product. Considerable support for this proper understanding of a press release may be found in the GAO draft report itself. The GAO draft report describes in significant detail the review, approval, and dissemination process for BJS reports. The draft report also describes a significantly different review, approval, and dissemination process for BJS press releases. The processes are different and the offices and staff involved are different because a press release is distinctly different from a statistical product. GAO's assumption that a press release is a statistical product has important ramifications that need to be kept in mind as one reads the draft report. Of primary concern is that the draft report overstates its findings relating to the potential threats to the independence of BJS because, the draft report often utilizes the broad term, "statistical products," when in many cases it is speaking only of press releases. This lack of distinction between the two terms should be kept in mind by the reader, especially in regard to the GAO finding that "the potential exists for BJS's review, approval, and dissemination process for statistical products to be subject to political interference." This imprecise statement is a focused reference to one press release, rather than a broader statement regarding BJS reports and other data products. As we state below, OMB, DOJ, OJP, and BJS all considered this exact question - whether or not a press release falls under guidelines for the dissemination of information - as these entities promulgated their information quality guidelines that govern information disseminated to the public. Each of these four executive branch agencies came to the same conclusion: press releases are exempt from information quality guidelines. See, e.g., Office of Management and Budget Information Quality Guidelines, I(A)(10)(October 1, 2002), "The guidelines focus on procedures for the `dissemination' of `information,' as those terms are defined herein. Accordingly, procedures specifically applicable to forms of communication outside the scope of these guidelines, such as those ,for correspondence or press releases, among others, are not included." (emphasis added). As the GAO draft report states, there was significant ambiguity on this point in the NRC guidelines as they stood in 2001. For at least four reasons, it is the view of OJP that it is not appropriate to apply the NRC guidelines to BJS press releases: 1) The text of two of the three guidelines at issue refer specifically to "data."[Footnote 24] For these guidelines to apply to press releases, the premise must be accepted that a press release belongs within the definition of "data." For the reasons discussed above, this assumption is unwarranted and thus we do not accept it. A fair reading of the purpose of these two guidelines is to ensure that a federal statistical agency control the data that it collects, analyzes, and releases to the public. OR affirms that BJS fully controls the hundreds upon thousands of spreadsheets, data sets, statistical reports, data briefs, and other statistical products that it collects, analyzes, and releases to the public. The text of the last guideline refers to the "release of statistical information."[Footnote 25] Notwithstanding the fact that this guideline is largely redundant to one referenced above, as OR has explained to GAO, BJS does have full authority to release statistical information without prior clearance, and BJS exercises this authority on a routine basis to present the public with a tremendous volume of criminal justice statistics in various formats without prior clearance. 2) When the press release for the 1999 PPCS was issued in 2001, there was no written or other guidance from NRC supporting GAO's position that the NRC guidelines covered press releases. Further, as of the date of this draft report, there have been three editions of Principles and Practices for a Federal Statistical Agency (1992, 2001, 2005), and not one has stated that the documents were intended to apply to press releases. GAO's effort in the draft report to cite to NRC's current interpretation of its guidelines is not persuasive. The relevant issue here relates not to whatever current advisory opinions might be provided to GAO by NRC staff, but whether the NRC written guidance available in 2001 explicitly covered press releases, and GAO and OJP both agree that it did not. 3) Despite OJP's efforts to explain this point to GAO in written and verbal discussions, the draft GAO report mischaracterizes the BJS Data Quality Guidelines as applying to press releases. OR reiterates that the BJS Data Quality Guidelines apply only to the statistics contained in BJS press releases, not to the entire document. This is because BJS accepts, as it must under the law, the judgments of OMB, DOJ, and OJP that press releases are outside the scope of these agencies' information quality guidelines. The BJS Data Quality Guidelines, cited by GAO, specifically state that they incorporate the exceptions found in the DOJ information quality guidelines; therefore, press releases are not included. 4) By statute, 42 U.S.C. 3732(b), the Director of BJS "shall be responsible for the integrity of data and statistics." In the exercise of such authority, he may elect to follow the NRC guidelines, but he is not and cannot be legally bound to do so, in the absence of some supervening statute (in the nature of Pub. L. No. 106-554). We are aware of no such statute relating to NRC; thus, even if the NRC written guidelines did apply to press releases (and they do not), the Director would and does decline, in the exercise of his statutory authority to apply them to BJS press releases. We are pleased that GAO has, through this effort, highlighted the importance of safeguarding the integrity of Federal statistical data. Because there are competing interpretations of what properly constitutes a "statistical product," we believe that the Federal statistical community would benefit from clarity in this area. OR and BJS will continue their long-standing commitment to integrity and objectivity in producing impartial and accurate statistics. Again, OR appreciates the opportunity to comment on this draft report. Sincerely, Signed by: Regina B. Schofield: Assistant Attorney General: [End of section] Appendix VII: GAO Contact and Acknowledgments: GAO Contacts: Brian J. Lepore (202) 512-4523: Acknowledgments: In addition to the above, Evi. L. Rezmovic, Assistant Director; Ronald S. Fecso, Chief Statistician; Jared A. Hermalin; Karen A. Jarzynka; Amanda K. Miller; Amy L. Bernstein; Geoffrey R. Hamilton; Robert Alarapon; and Tracy J. Harris made key contributions to this report. FOOTNOTES [1] At the time of our review, BJS had not issued any products from the 2005 survey. [2] Of the consolidated list of 24 review, approval, and dissemination guidelines issued by the National Research Council, Department of Justice, Office of Justice Programs, and Bureau of Justice Statistics, only 10 were available in 2001, when the press release based on the 1999 survey was issued. [3] GAO, Data Quality: Expanded Use of Key Dissemination Practices Would Further Safeguard the Integrity of Statistical Data, GAO-06-607 (Washington, D.C.: May 31, 2006). [4] Pub. L. No. 96-157, 93 Stat. 1167 (1979). [5] Office of Management and Budget, Statistical Programs of the United States Government: Fiscal Year 2006, pp. 3-4 (Washington, D.C.: 2005). [6] Pub. L. No. 103-322, 108 Stat. 1796, 2071 (1994) (codified at 42 U.S.C. 14142). [7] For ease of presentation, in this report we use the term "guidelines" to refer to the principles and practices described in the National Research Council document, as well as to guidelines issued by the Department of Justice, Office of Justice Programs, and BJS. [8] The National Research Council's Committee on National Statistics was established in 1972 at the recommendation of the President's Commission on Federal Statistics in order to improve the statistical methods and information on which public policy decisions are based. [9] Margaret E. Martin, Miron L. Straf, and Constance F. Citro (eds.), Principles and Practices for a Federal Statistical Agency, (3rd ed.) (Washington, D.C.: The National Academies Press, 2005). [10] Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Pub. L. No. 106-554, 114 Stat. 2763, 2763A- 125 (2000)), known as both the Information Quality Act and the Data Quality Act, required OMB to issue governmentwide quality guidelines including the requirement that such agencies issue their own quality guidelines. [11] See Notice of revisions to OMB's Standards and Guidelines for Statistical Surveys, 71 Fed. Reg. 55522 (Sept. 22, 2006). [12] After OMB issued policy and procedural guidance to federal agencies in 2002, the Department of Justice, Office of Justice Programs, and BJS developed their own guidelines. [13] Information posted on the BJS Web site includes hypertext links to information that is created and maintained by other public and private organizations in the United States and by other nations. [14] Guidelines developed by BJS, the Office of Justice Programs, and the Department of Justice were not available at the time the press release, based on the 1999 survey, was released. [15] Our work focused exclusively on the Police-Public Contact Survey, and we did not conduct a systematic review of other press releases that have been issued based on BJS's work, to make an independent determination of whether they constituted statistical products. [16] See GAO-06-607. [17] OMB's February 2002 guidelines for federal agencies exempted press releases from the guidelines. These guidelines, however, were designed to apply to a wide variety of government information dissemination activities, and be generic enough to fit all types of media. As we noted in our May 2006 report (GAO-06-607), the directive on which OMB is currently working pertains directly to the release and dissemination of statistical products. [18] We did not review the extent to which BJS followed guidelines for the 2005 draft press release (based on the 2002 survey) that was the source of disagreement between the then-BJS Director and the then- Acting Assistant Attorney General, because the press release was not issued. [19] U.S. Code, Title 42, Chapter 46, Subchapter I (42 U.S.C. Sections 3711 through 3715a). [20] In contrast to the process followed for press releases, the only noncareer appointee involved in the review, approval, and dissemination of reports from Police-Public Contact Surveys is the BJS Director. Appendix IV provides details on the roles of the Director and other employees in this process. [21] The number of guidelines that were applicable to the review, approval, and dissemination of Police-Public Contact Surveys products varied across the time periods covered. In general, the number of guidelines increased between February 2001, when the first survey report was issued, and June 2006, when the latest survey report was issued. However, one National Research Council guideline, pertaining to the dissemination of data and information to as broad a public as reasonably possible, was dropped when the council issued the second edition of its guidelines between the time that BJS issued its first report (in February 2001) and second report (in March 2002) from the 1999 survey. [22] This included persons living in group quarters, but excluded institutionalized persons and some members of the armed services. [23] Principles and Practices for a Federal Statistical Agency, National Research Council, Committee on National Statistics, National Academy Press, p. 4 (1992)("Some aspects of [agency] independence, not all of which are required, are the following,"). [24] As stated in the draft report, the guidelines read (emphases added): "The statistical agency has authority for professional decisions over the scope, content, and frequency of data compiled, analyzed, and published." "The release of information should not be subject to actual or perceived political interference. In particular, the timing of the public release of data should be the responsibility of the statistical agency." [25] As stated in the draft report, the guideline reads, "The statistical agency has recognition by policy officials of its authority to release statistical information without prior clearance." GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to www.gao.gov and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, D.C. 20548: Public Affairs: Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.