Rental Housing

HUD Can Improve Its Process for Estimating Fair Market Rents Gao ID: GAO-05-342 March 31, 2005

The Department of Housing and Urban Development (HUD) annually estimates fair market rents (FMR) for standard quality rental units throughout the United States. Among other uses, FMRs help determine subsidies for almost 2 million low-income families in the nation's largest rental assistance program. However, concerns exist that FMRs can be inaccurate--often, too low, preventing program participants from finding affordable housing. Also, HUD will soon derive FMRs from a new source, the American Community Survey (ACS), which processes data somewhat differently than HUD's current data sources, including the decennial census. You asked us to review (1) how HUD estimates FMRs, (2) how accurate FMRs have been, (3) how ACS data may affect accuracy, and (4) other changes HUD can make to improve the estimates.

According to HUD, the typical process for estimating FMRs includes benchmarking, or developing baseline rents for each FMR area (generally county-based) using census data or other surveys for the years between censuses; adjusting those rents to bring them up to date; and seeking public comment before finalizing the numbers. HUD generally uses Consumer Price Index and telephone survey data to adjust baseline rents--that is, to account for rent changes since data used for baseline estimates were collected and to project the estimates into the next fiscal year (when they will be in use for subsidy purposes). HUD then lists the proposed FMRs in the Federal Register for public comment. These comments can lead to changes in FMRs, but only when they include new data or lead HUD to conduct a new survey. About 69 percent of all areas had FMR estimates in use in 2000 that were within 10 percent of rents indicated by the 2000 decennial census--the most accurate comparison data available for each FMR area. This represents an improvement over HUD's 1990 estimates. Similarly, about 73 percent of 153 areas whose FMRs HUD rebenchmarked after 2000 were within 10 percent of rents derived from recent surveys. In general, GAO found that areas that are rebenchmarked with more recent data tended to have FMRs in the most accurate range (within 10 percent). Using ACS data could improve the accuracy of FMRs by allowing HUD to benchmark more areas more frequently than is possible with current data sources, using more recent data--a factor that GAO's analysis suggests is related to accuracy. HUD's first use of ACS data will be to update existing baseline estimates for the fiscal year 2006 FMRs; HUD expects to use ACS data to set baseline rents for some fiscal year 2008 FMRs. HUD could improve its FMR estimation process by consistently following its guidelines relating to the transparency of FMRs and ensuring that it can assess the accuracy of ACS-based FMRs. Transparency would be improved by fully documenting the estimation process so that FMRs can be independently reproduced. Even ACS-based FMRs may not always be accurate, and HUD's policies require mechanisms to correct information it disseminates.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-05-342, Rental Housing: HUD Can Improve Its Process for Estimating Fair Market Rents This is the accessible text file for GAO report number GAO-05-342 entitled 'Rental Housing: HUD Can Improve Its Process for Estimating Fair Market Rents' which was released on May 2, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Ranking Minority Member, Subcommittee on Housing and Transportation, Committee on Banking, Housing, and Urban Affairs, U.S. Senate: March 2005: Rental Housing: HUD Can Improve Its Process for Estimating Fair Market Rents: [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-342]: GAO Highlights: Highlights of GAO-05-342, a report to the Ranking Minority Member, Subcommittee on Housing and Transportation, Committee on Banking, Housing, and Urban Affairs, U.S. Senate: Why GAO Did This Study: The Department of Housing and Urban Development (HUD) annually estimates fair market rents (FMR) for standard quality rental units throughout the United States. Among other uses, FMRs help determine subsidies for almost 2 million low-income families in the nation‘s largest rental assistance program. However, concerns exist that FMRs can be inaccurate”often, too low, preventing program participants from finding affordable housing. Also, HUD will soon derive FMRs from a new source, the American Community Survey (ACS), which processes data somewhat differently than HUD‘s current data sources, including the decennial census. You asked us to review (1) how HUD estimates FMRs, (2) how accurate FMRs have been, (3) how ACS data may affect accuracy, and (4) other changes HUD can make to improve the estimates. What GAO Found: According to HUD, the typical process for estimating FMRs includes benchmarking, or developing baseline rents for each FMR area (generally county-based) using census data or other surveys for the years between censuses; adjusting those rents to bring them up to date; and seeking public comment before finalizing the numbers. HUD generally uses Consumer Price Index and telephone survey data to adjust baseline rents”that is, to account for rent changes since data used for baseline estimates were collected and to project the estimates into the next fiscal year (when they will be in use for subsidy purposes). HUD then lists the proposed FMRs in the Federal Register for public comment. These comments can lead to changes in FMRs, but only when they include new data or lead HUD to conduct a new survey. About 69 percent of all areas had FMR estimates in use in 2000 that were within 10 percent of rents indicated by the 2000 decennial census”the most accurate comparison data available for each FMR area. This represents an improvement over HUD‘s 1990 estimates, as the table below shows. Similarly, about 73 percent of 153 areas whose FMRs HUD rebenchmarked after 2000 were within 10 percent of rents derived from recent surveys. In general, GAO found that areas that are rebenchmarked with more recent data tended to have FMRs in the most accurate range (within 10 percent). Using ACS data could improve the accuracy of FMRs by allowing HUD to benchmark more areas more frequently than is possible with current data sources, using more recent data”a factor that GAO‘s analysis suggests is related to accuracy. HUD‘s first use of ACS data will be to update existing baseline estimates for the fiscal year 2006 FMRs; HUD expects to use ACS data to set baseline rents for some fiscal year 2008 FMRs. HUD could improve its FMR estimation process by consistently following its guidelines relating to the transparency of FMRs and ensuring that it can assess the accuracy of ACS-based FMRs. Transparency would be improved by fully documenting the estimation process so that FMRs can be independently reproduced. Even ACS-based FMRs may not always be accurate, and HUD‘s policies require mechanisms to correct information it disseminates. Accuracy of HUD‘s Fiscal Years 2000 and 1990 FMR Estimates: [See PDF for image] Sources: GAO analysis of HUD data (2000 figures) and HUD (1990 figures). [End of table] What GAO Recommends: To improve the usefulness of its FMR estimates, GAO recommends that HUD fully document its methods for estimating FMRs by following all of its data quality guidelines; use, to the extent possible, state-level ACS data to update the fiscal year 2006 FMRs; and develop a mechanism to assess the accuracy of future FMRs. In response to a draft of this report, HUD agreed to better document methods for estimating FMRs and said it is exploring options to assess accuracy. www.gao.gov/cgi-bin/getrpt?GAO-05-342. To view the full product, including the scope and methodology, click on the link above. For more information, contact David G. Wood at (202) 512-8678 or woodd@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: HUD Estimates FMRs by Defining Housing Markets, Choosing Data Sources, Updating Rent Data, and Evaluating Public Input: Most FMR Estimates Were Accurate within 10 Percent of the Census or Other Rebenchmarking Surveys: ACS Could Improve the Accuracy of FMRs by Providing HUD with More Recent, Better Data: HUD Did Not Follow One of Its Data Quality Guidelines and May Lack Data Sources to Assess the Accuracy of Future FMRs: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendixes: Appendix I: Objectives, Scope, and Methodology: Appendix II: Comments from the Department of Housing and Urban Development: Appendix III: GAO Contacts and Staff Acknowledgments: GAO Contacts: Staff Acknowledgments: Tables: Table 1: Accuracy of HUD's Fiscal Years 2000 and 1990 FMR Estimates Compared with Rents from Census: Table 2: Accuracy of HUD's FMR Estimates Compared with Rents from RDD Surveys (by Reason for Survey, 2001-05): Table 3: Accuracy of FMR Estimates in 2000 Compared with Rents from Census (Based on Age of Baseline FMR Data): Table 4: Accuracy of FMR Estimates in 2000 Compared with Rents from Census (Based on Type of Rebenchmarking Survey): Table 5: Accuracy of FMR Estimates in 2000 Compared with Rents from Census (by Type of Update Factor): Figures: Figure 1: Example of 40th Percentile of Rent: Figure 2: HUD's Typical Process for Estimating FMRs12: Figure 3: HUD Regions: Figure 4: Accuracy of HUD's Fiscal Year 2000 FMR Estimates: Figure 5: Scope of ACS Rebenchmarking as Related to FMR Area Size and Housing Choice Voucher Program Data: Abbreviations: ACS: American Community Survey: AHS: American Housing Survey: BAH: basic allowance for housing: CPI: Consumer Price Index: DOD: Department of Defense: FMR: fair market rent: HOPWA: Housing Opportunities for Persons with AIDS: HUD: Department of Housing and Urban Development: LIHTC: Low Income Housing Tax Credit: NAS: National Academy of Sciences: OMB: Office of Management and Budget: PHA: public housing agency: RDD: random digit dialing: SRO: Moderate Rehabilitation Single-Room Occupancy: Letter March 31, 2005: The Honorable Jack Reed: Ranking Minority Member: Subcommittee on Housing and Transportation: Committee on Banking, Housing, and Urban Affairs: United States Senate: Dear Senator Reed: The Department of Housing and Urban Development's (HUD) Housing Choice Voucher Program, commonly known as "Section 8" tenant-based assistance, is the largest ongoing rental assistance program in the United States, serving almost 2 million families with a budget of about $16.9 billion for fiscal year 2005. The Housing Choice Voucher Program provides subsidies to help low-income families afford rental housing in the private market. To determine the amounts of the subsidies it will provide to low-income families under the Voucher Program, and for other purposes, HUD annually estimates fair market rents (FMR)--that is, rent estimates that include utilities. From time to time, public housing agencies and others have expressed concern that FMR estimates can be inaccurate--often, too low--thereby preventing voucher holders from being able to find affordable housing in certain areas. HUD estimates FMRs for all bedroom size units for each area in the entire United States (typically, counties) in advance of the year during which they will be effective. HUD currently uses rent data from a variety of surveys--the Bureau of the Census' decennial census long form is the major survey used--as a baseline (or benchmark) for estimating FMRs throughout the country.[Footnote 1] Between censuses, HUD's practice has been to rebenchmark census-based FMRs with data from the American Housing Survey (AHS), a Census Bureau survey performed in certain metropolitan areas every few years, and from Random Digit Dialing (RDD) surveys, telephone interviews that gather rent and other data for estimating FMRs for a limited number of metropolitan and nonmetropolitan areas annually, conducted by HUD contractors. However, a new Census Bureau product, the American Community Survey (ACS), is replacing the decennial census long form and will become the major source of rent data for FMR estimates in every area. With the ACS, the Census Bureau will publish results annually based on 1-, 3-, or 5-year averages, depending on the population size of the area surveyed, rather than every 10 years. For example, HUD will receive 1-year average data (the average of 12 months) annually for areas in which the majority of voucher holders reside. You asked us to review HUD's process for estimating FMRs and the impact that the incorporation of the ACS could have on the accuracy of FMRs. Our report discusses (1) how HUD estimates FMRs, (2) how accurate HUD's FMR estimates have been, (3) how and when the use of ACS data to estimate FMRs may affect their accuracy, and (4) the potential for other changes HUD could make to improve the way it estimates FMRs and their accuracy. To determine the general process for how HUD estimates FMRs, we analyzed statutes, regulations, and agency documents and interviewed HUD officials. To determine how accurate FMR estimates were, we compared all two-bedroom FMRs that HUD put in effect for fiscal year 2000 with census data for the same year because (1) the decennial census rent estimates are considered to be the closest estimates of the true value of those rents and (2) HUD estimates FMRs for other bedroom sizes as a multiple of the FMR it sets for two-bedroom units. We also compared HUD's estimated FMRs in effect during fiscal years 2001-05 for selected geographic areas with rents estimated using data from surveys HUD and others conducted over this period. After making these comparisons, we performed an associative analysis--that is, we analyzed specific components of (or data inputs to) the FMR estimation process to see how they might relate to the accuracy of FMRs. To determine how and when HUD will use ACS data to estimate FMRs and what their potential effects on the accuracy of FMRs would be, we compared ACS with the other major surveys HUD uses to estimate FMRs, identified salient characteristics of the ACS data, and reviewed HUD's plans for using ACS data. To determine other changes HUD could make to improve its estimation process and the accuracy of FMRs, we analyzed data quality guidelines and then assessed HUD's estimation process against the guidelines. We also interviewed officials from HUD headquarters and field offices, as well as experts and researchers who routinely work with housing data sources. Appendix I provides additional details on our objectives, scope, and methodology. Throughout this report, we refer to the "quality" of surveys or the "quality" of data. We use quality as an overarching term for important characteristics related to the accuracy, recency, and relevance of data sources and surveys. Specifically, for purposes of this report, we describe quality data obtained from surveys as: * "accurate" when all types of rental housing units have a chance of being selected for the survey and the sample size is large enough to provide a 90 or 95 percent likelihood that the survey's estimates will be within 5 to 10 percent of what would be found if the entire population (i.e., all rents) were known; * "recent" to the extent that the time between when data are collected and subsequently used is minimized; and: * "relevant" when surveys collect, at a minimum, data on rents for HUD's program purposes and, among the survey data sources available, HUD chooses the survey that most closely corresponds to the FMR area. These characteristics generally match those in data quality guidelines used by other federal agencies, and the characteristics of data or survey quality required by HUD through statute, regulations, or guidance for data submissions. We conducted our work in Washington, D.C., between May 2004 and February 2005 in accordance with generally accepted government auditing standards. Results in Brief: According to HUD, the typical process to estimate FMRs includes developing baseline rents from what it judges to be the best rent data available for each area, adjusting those rents to bring them up to date, and seeking public comment on its estimates prior to publishing them for public housing agencies and others to use. Once HUD determines the FMR areas, it uses decennial census housing data when they are first released to establish baseline rent estimates, or benchmarks, for each. For subsequent years, HUD uses data from other surveys--either the AHS or RDD surveys--to establish a new baseline, or to "rebenchmark" FMRs for certain areas. To compensate for the time lag between when data are collected and when HUD first uses them, HUD annually adjusts its baseline estimates in two ways. First, HUD updates the estimates to December 31 of the current fiscal year using annual percentage changes in rent and utility costs from the local Consumer Price Index for major metropolitan areas, or similar information from RDD surveys for other areas. Second, to make FMRs relevant for the fiscal year in which they will be in effect, HUD trends, or projects, the updated figure to the midpoint of the next fiscal year by applying a national estimate of annual rent increases between the censuses from the decennial census data. After making these adjustments, HUD publishes the proposed FMRs in the Federal Register for public comment. Although HUD considers all of the comments it receives, it typically changes the proposed FMRs only if the comments are supported by data that meet HUD's standards. After the period of 60 days to comment on the Federal Register ends, HUD still considers other requests and submissions throughout the year. Over two-thirds of FMRs that HUD estimated for fiscal year 2000, as well as those it estimated for areas rebenchmarked after 2000, were within 10 percent of the rents indicated by a subsequent quality survey, such the AHS. For example, when we compared the fiscal year 2000 FMRs (which HUD estimated in 1999) with rents from the 2000 census data that were collected during the same period the FMRs were in effect, 69 percent of all of HUD's FMR area estimates were within 10 percent of the census figure--an improvement over HUD's 1990 estimates, when 39 percent of areas were within 10 percent of the 1990 census. When we compared a limited number of FMRs that HUD estimated after 2000 with rents indicated by data from the AHS or RDD surveys that HUD or public housing agencies (PHA) subsequently conducted, a similar proportion of FMRs (73 percent) fell in the most accurate range. While our associative analysis did not demonstrate what factors definitively cause accuracy or how much each contributes, it did show that when HUD used more recent, relevant data taken from a higher quality survey than some HUD used to rebenchmark in the past, FMR estimates were more often within 10 percent of the rents derived from a rebenchmarking survey. For example, FMR estimates from areas based on more recent survey data- -within 1 to 4 years--produced a significantly higher proportion of FMRs that were within 10 percent of rents derived from the census than FMR estimates from areas surveyed less recently. The ACS could improve the accuracy of FMR estimates because it is a higher quality survey than some HUD has used in the past and provides more recent and local data than are currently available--beginning in fiscal year 2006 when HUD first uses ACS data to update FMRs, and subsequently in fiscal year 2008 when it will likely rebenchmark FMRs in certain areas. HUD will be able to use ACS data to rebenchmark FMRs annually (or every 3 or 5 years for areas with smaller populations), doing so in generally the same way it used the decennial census to estimate baseline rents. Certain challenges related to the manner in which ACS data are processed and reported may affect FMR accuracy. For example, ACS data are averages of monthly survey data, which may "smooth" rental market shifts or trends. According to HUD officials, they will begin to address these challenges when the Census Bureau releases the fiscal year 2005 data (in Fall 2006), the data collected during the first year of full implementation for the ACS. Despite the challenges in using the data, neither we nor experts and researchers who routinely work with housing data sources identified viable alternatives to the ACS. Potential exists for HUD to improve its estimation process for FMRs and their accuracy because the agency (1) presently does not follow its objectivity guideline for ensuring the transparency and reproducibility of its FMR estimates and (2) may in the future lack a way to assess the accuracy of ACS-based rent estimates. HUD, like other federal agencies, has developed guidelines to ensure that it disseminates quality data. HUD's guidelines include ensuring the utility (usefulness), integrity (protection from unauthorized access), and objectivity (transparency and reproducibility) of data. Of the three, HUD appears to be following the utility and integrity guidelines as they relate to the FMR estimation process. For example, HUD meets its utility guidelines by estimating FMRs on an annual schedule and making the estimates public and easily accessible. HUD does not follow one of these three--its objectivity guideline--because it has made neither the data it uses nor its methods for estimating FMRs sufficiently transparent for an independent party, such as GAO, to be able to substantially reproduce FMRs using publicly available information. Finally, as HUD transitions to ACS-based FMRs, it will not only stop using the decennial census long form but it will rely less on RDD surveys and the AHS because of cost and quality concerns about these surveys. As a result, HUD may not have a means to assess the accuracy of future FMR estimates once it relies almost exclusively on the ACS. This report contains recommendations designed to improve HUD's processes for estimating FMRs and their accuracy. We provided HUD with a draft of this report for its review and comment. HUD agreed that it can better document its methods for estimating FMRs and described efforts it has under way to improve the transparency and reproducibility of its methods. HUD also requested that we clarify certain transparency and reproducibility issues in our report and recognize its ongoing efforts. HUD disagreed with our recommendation to use state-level ACS data in fiscal year 2006 FMRs, stating that it has concerns about the adequacy of ACS sample sizes for the fiscal year 2006 estimates. We have retained this recommendation because it contains a caution that HUD should do so as much as possible, but only in instances where HUD determines that the ACS data are sufficiently reliable for this purpose. HUD did not explicitly state whether it agrees or disagrees with our recommendation that it develop a mechanism to assess the accuracy of future FMRs, but it did indicate that it recognizes there are areas, such as those with unusual rent increases or decreases, that could experience FMR estimation errors when HUD uses ACS data for its estimates. HUD also indicated that it anticipates continuing to review AHS surveys and making limited use of RDD surveys while it explores other long-term alternatives for assessing the accuracy of FMRs. Because HUD recognized the challenge we pointed out relating to the accuracy of FMRs and stated that it is currently exploring ways to address this issue, we have retained our recommendation. HUD also suggested a number of technical clarifications to our report, which we have made, as appropriate. Background: HUD estimates FMRs in order to set upper and lower bounds on the cost and quality of typical, standard quality units voucher holders rent and, in doing so, ensure that the units rented are modest (not luxurious), meet the housing quality standards HUD sets for them, and are available in sufficient numbers to those seeking to use the vouchers. Local PHAs use FMRs to set payment standards, which are the basis for determining the subsidies HUD provides to help low-income families afford housing in the private rental market under the Housing Choice Voucher Program. Specifically, PHAs may set payment standards at 90 to 110 percent of the FMR for their area and, with HUD approval, above 110 percent of the FMR. Because HUD generally requires voucher holders to contribute 30 percent of their income as rent, the amount of HUD's subsidy (the rental assistance) then becomes the difference between the PHA's payment standard and 30 percent of the family's monthly income.[Footnote 2] While FMRs are primarily used in the Housing Choice Voucher Program, other programs both inside and outside of HUD also use FMRs. For example, HUD uses FMRs to: * determine initial rents for housing assistance payments in the Moderate Rehabilitation Single-Room Occupancy program;[Footnote 3] * determine initial renewal rents for units in some expiring project- based "Section 8" contracts under the Mark-to-Market Program;[Footnote 4] * set maximum rents under the HOME Program;[Footnote 5] * set standard rent ceilings in the Housing Opportunities for Persons with AIDS (HOPWA) Program;[Footnote 6] * make calculations for the "difficult development" areas under the Low Income Housing Tax Credit (LIHTC) Program;[Footnote 7] and: * review the feasibility of proposed LIHTC projects. The Department of Defense (DOD) compares its basic allowance for housing (BAH) amounts, which is housing assistance it provides military personnel, to HUD's FMRs. More specifically, when DOD determines that it is not cost-effective to collect proprietary survey data on housing costs, it uses FMRs as a basis for calculating comparable figures. Whatever its programmatic use, an FMR must fall within certain statutory and regulatory parameters. The U.S. Housing Act of 1937, as amended, requires HUD to base FMRs on the most recent available data to estimate rents of various sizes and types within a market.[Footnote 8]HUD regulations and guidance on FMRs further emphasize that rent survey data must be the most accurate and current available.[Footnote 9] HUD specifically requires that the survey methodology provide statistically reliable, unbiased estimates of gross rents by, among other things, having a large enough sample so that there is a 95 percent likelihood that the survey's estimates will be within 5 to 10 percent of what would be found if the entire population (i.e., all rents) were collected. HUD also requires that survey samples be random and reflect rent levels that exist for housing units of different ages, types, and geographic locations within the entire FMR area. Using these considerations, HUD's three primary data sources for FMRs are the decennial census (long form), the AHS, and RDD surveys. A RDD survey is a computer-aided telephone survey of randomly selected households that may be conducted by HUD, individual PHAs, or others. Finally, FMRs are specifically defined as annual estimates of the 40th percentile of gross rents for typical, nonsubstandard market-rate rental units occupied by recent movers.[Footnote 10] Fortieth Percentile of Rents: The 40th percentile is the point in a distribution of numbers at which 40 percent of the numbers are at or below that point; for FMR purposes, this is the dollar amount below which 40 percent of the standard quality rental units in an area have rented. For example, in the distribution in figure 1, $670 is the 40th percentile because 4 of the 10 rents are at or below that point: Figure 1: Example of 40th Percentile of Rent: [See PDF for image] [End of figure] Gross Rent: A gross rent is the rent a tenant pays to the owner--sometimes called "shelter" costs--plus the cost of utilities (usually, electricity, gas, water and sewer, and trash removal charges, but not telephone service). If utilities are included in the rent, then the gross rent is simply the amount paid to the owner. Typical, Standard Rental Units: By statute, FMRs are estimates of market rents for typical, standard quality housing. HUD has determined that certain rental units should be excluded from its data sources in order to meet this definition. Specifically, these include rents for units built within the last 2 years (which tend to be higher priced); units receiving some form of subsidy (such as public housing) where the rent does not reflect a "market" price; and substandard units--for example, units without adequate heating or plumbing--that likely would not meet the housing quality standards applicable to the voucher program.[Footnote 11] Recent Movers: HUD has found that rents for units occupied by recent movers (i.e., tenants who moved within the past 15 to 24 months) are typically higher than what other renters pay. By linking FMR estimates to the rents that recent movers have paid, HUD tries to ensure that they more closely reflect the rents that low-income households new to the voucher program may face when they look for rental housing. The Census Bureau is discontinuing the long form and has begun replacing it with the ACS.[Footnote 12] Overall, the ACS will provide the same type of data as the decennial census long form at the same level of geographic area detail, but in a more timely way because it will be an ongoing survey (as opposed to one conducted every 10 years). Specifically, the ACS will collect data monthly and each year publish either 1-, 3-, or 5-year averages (depending on the population in each area).[Footnote 13] HUD Estimates FMRs by Defining Housing Markets, Choosing Data Sources, Updating Rent Data, and Evaluating Public Input: According to HUD, the typical process it uses to estimate FMRs (rent estimates that include utilities) includes choosing what it judges to be the best rent data available for each area, adjusting those data so that they are up to date, and seeking public comment on the estimates prior to finalizing them for public housing agencies and others to use (see fig. 2). Once HUD determines each FMR area and receives decennial census data or AHS or RDD data, it analyzes the rent data to establish a "benchmark" FMR for each area by determining the 40th percentile of the rent distribution. Then, HUD annually adjusts the estimates to reflect changes in rent and utility costs to compensate for the lag between data collection and the period in which the FMR will be in effect. After adjusting the FMR for each area, HUD publishes the proposed FMRs for public comment. Although HUD considers all of the comments it receives, it typically changes FMRs only if the comments are supported with data that meet HUD's standards. The public can also affect FMRs by (1) requesting that HUD conduct an RDD for the area or (2) submitting comments with supporting rent data or information that causes HUD to conduct additional research. Figure 2: HUD's Typical Process for Estimating FMRs: [See PDF for image] [End of figure] HUD Establishes Areas, Uses Survey Data to Benchmark and Adjust FMRs: To ensure that the FMR estimates are useful, HUD's first step is to determine FMR areas that they believe correlate with distinct housing markets, typically the size of a county (see fig. 2). To determine FMR areas, HUD generally uses the boundaries of Office of Management and Budget (OMB)-defined metropolitan and nonmetropolitan areas.[Footnote 14] According to HUD, it may also create new areas that do not correspond to OMB boundaries, particularly within sprawling metropolitan areas that may have separate housing markets. For instance, HUD created a separate FMR area for West Virginia counties that had been included in OMB's Washington, D.C., metropolitan area, because HUD did not consider these counties to be part of the Washington housing market. Although HUD may revise FMR area definitions at any time, it typically does so infrequently (not every year when it develops FMRs).[Footnote 15] HUD publishes FMR estimates annually for 356 metropolitan FMR areas and 2,303 nonmetropolitan FMR areas in the United States, Puerto Rico, the Virgin Islands, and Guam. HUD's second step is to benchmark--that is, estimate baseline rents-- for two-bedroom units by identifying the 40th percentile of the estimated rent distribution for each area with the most recent available data (for FMR areas for which no new, recent rent data are available, HUD skips this step and updates the existing FMR). HUD chooses from a variety of data for benchmarking, including the decennial census, the AHS, RDD surveys, and traditional surveys from the public. According to HUD officials: * The decennial census provides the highest quality data to estimate FMRs because it provides (1) rent estimates within 1 percent of the true value of the 40th percentile of rents in metropolitan areas and (2) the most consistent data for all areas to establish a baseline for FMRs once every 10 years. * Data from RDD surveys have sufficient quality to meet HUD's requirements and provide estimates within 3.5 to 5 percent of the true value of rents for a limited number of areas, usually metropolitan areas. * The AHS offers sufficient quality data with estimates within 7 percent of the true value of rents the survey is measuring and are available for a limited number of metropolitan areas every few years. According to HUD officials, to be consistent with the definition of FMRs, HUD only uses survey data for rental housing units that are: * nonsubsidized and of "standard" quality;[Footnote 16] * more than 2 years old; * nonseasonal (i.e., occupied year round); * located on properties of less than 10 acres; and: * leased by recent movers (those who have moved within the last 15 to 24 months). HUD adds estimated utility costs to the base rent estimates it derives from RDDs because these surveys do not include that information. To do so, HUD officials estimate the cost of utilities with PHA utility schedules, which include a list of average monthly costs for each utility. The decennial census and AHS data include utilities in their base year rent estimates. The third and fourth steps in the process involve adjusting FMRs. To mitigate the time lag between data collection and FMR use, HUD first updates FMRs to December 31 of the current fiscal year with information about changes in the rent and utility index from the Consumer Price Index (CPI) program for specific metropolitan areas or, for other metropolitan and all nonmetropolitan areas, with the gross rent "change factors" established by regional RDD surveys. To estimate the gross rent change factor, or the measure of rent change, HUD conducts regionwide RDD surveys in each of its 10 multistate regions (see fig. 3). Figure 3: HUD Regions: [See PDF for image] [End of figure] Once FMRs are updated, HUD attempts to make them useful for the fiscal year in which they will be in effect by trending, or projecting, them to the midpoint of that fiscal year. To do this, HUD uses a national measure of annual rent increases (i.e., average rent increases during the 10 years between the censuses, typically 3 percent, on the basis of decennial census rent data). In the fifth step, HUD also estimates FMRs for other bedroom sizes (in practice, one-, three-, and four-bedroom). Because HUD usually lacks sufficient survey data to directly estimate FMRs for all unit sizes, it typically benchmarks FMRs for two-bedroom units only and estimates rent ratios for other sizes.[Footnote 17] According to HUD officials, these ratios are based on local rent relationships derived from decennial census rent data. Once HUD calculates these ratios, it ensures that they are "sequential," which means that FMRs increase as unit size increases (e.g., in 1994, three-bedroom FMRs had to be at least 125 percent of two-bedroom FMRs, and four-bedroom FMRs had to be at least 140 percent of two-bedroom FMRs). After HUD estimates FMRs for each bedroom size unit, it applies a "bonus" to increase FMRs for larger units (three-bedrooms or larger) to help ensure that the units can be rented by voucher holders. HUD Provides Opportunities for Public Input on Proposed FMRs as Well as Those in Effect: To provide for public input on proposed FMRs: * HUD publishes the proposed FMRs in the Federal Register to solicit public comments, usually in April or May of each year (sixth process step). * The public submits comments during the (approximate) 60-day public comment period. * After the comment period, HUD reviews the responses received and may act on some of them prior to finalizing FMRs and publishing them again in final form in the Federal Register in September (seventh process step). FMRs are in effect for the next fiscal year, which starts October 1. After the period of 60 days to comment on the Federal Register ends, to address situations in which existing FMRs are perceived to be inaccurate, members of the public--often, PHAs--also can submit information on the existing FMR for HUD to consider. For example, PHAs can at any time conduct and submit to HUD the results of their own RDD surveys; HUD applies the same criteria to these surveys as it does to those that PHAs submit in response to the proposed FMRs in the Federal Register. Specifically, HUD requires that any PHA-submitted data it uses to change FMRs must be statistically reliable; unbiased estimates of gross rents; and, among other things, have a large enough sample that there is a 95 percent likelihood that the survey's estimates will be within 10 percent of what would be found if the entire population (i.e., all rents) were collected.[Footnote 18] Also, PHAs may at any time outside of the formal comment process request that HUD conduct an RDD survey or submit information about the existing FMR that may cause HUD to conduct additional research. While the Quality Housing and Work Responsibility Act of 1998 gave PHAs the flexibility to set payment standards at 90 to 110 percent of their FMRs, they may also request an exception to further adjust either the payment standard or the FMR for their area. Specifically, when PHAs believe that payment standards at 110 percent of the FMR are insufficient to allow voucher holders to successfully lease units, they may request from HUD one of two possible exceptions: (1) increase the payment standard to exceed the FMR by more than 10 percent or (2) benchmark the FMR estimate at the 50TH percentile of rent for the area, rather than the 40th percentile of rent.[Footnote 19] Most FMR Estimates Were Accurate within 10 Percent of the Census or Other Rebenchmarking Surveys: According to our analysis, more than two-thirds of (1) all FMRs that HUD estimated for fiscal year 2000 and (2) a limited number of FMRs that HUD rebenchmarked after 2000 were within 10 percent of the rents derived from subsequent surveys such as the census, the AHS, or an RDD survey. Specifically, 69 percent of all of HUD's FMR estimates for fiscal year 2000--published in 1999--were within 10 percent (the most accurate range) of rent estimates derived from the 2000 census. Moreover, when we considered FMRs by type of area, FMR estimates for 86 percent of metropolitan areas and 66 percent of nonmetropolitan areas fell in the most accurate range in 2000. Similarly, when we compared rents derived from rebenchmarking surveys done for 153 FMR areas since 2000 with the FMR estimates in place at the time of the rebenchmarking survey, 73 percent of the estimates were within 10 percent of the rents derived from the surveys. FMR estimates were more often associated with accuracy when HUD based them on data that were more recent, taken from a higher quality survey than some HUD has used in the past, or more relevant because the survey covered an area closely matching the boundaries of the FMR area.[Footnote 20] Other factors not related to the specific survey HUD used to estimate FMRs, such as difficulty in estimating utility costs, may also affect the accuracy of FMR estimates. Over Two-thirds of All FMRs for 2000 Were Accurate within 10 Percent of Rents Derived from the 2000 Census: According to our analysis, for fiscal year 2000, 69 percent of FMRs that HUD estimated for fiscal year 2000 were within 10 percent of the 2000 census rent estimates, the most accurate comparison data available for each FMR area (see fig. 4). Figure 4: Accuracy of HUD's Fiscal Year 2000 FMR Estimates: [See PDF for image] [End of figure] FMR estimates that were within 10 percent of the rents derived from a higher quality survey, such as a census or RDD survey, could be higher or lower (i.e., within plus or minus 10 percent). For example, if HUD estimated an FMR of $500 for an area and a higher quality survey of the same area found a 40th percentile rent of $550, the difference is within 10 percent of the survey as follows: $550 (new survey estimate)-$500 (existing FMR) = $50 difference: $50 difference/$550 (new survey estimate) = 9 percent: In this example, the original FMR was lower than the rent indicated by the recent survey, but was within 10 percent. The results for 2000 are a significant improvement over results from 1990 when HUD reported that 39 percent of FMRs were in the most accurate range (see table 1). Furthermore, arraying the data by population to account for areas where estimates affected more potential voucher holders shows that a greater share of FMR estimates were within the most accurate range in 2000 than what HUD reported for 1990. Considering FMR estimates by type of area, we also found that more metropolitan and nonmetropolitan areas were within 10 percent accuracy in 2000 than HUD reported in 1990.[Footnote 21] Table 1: Accuracy of HUD's Fiscal Years 2000 and 1990 FMR Estimates Compared with Rents from Census: Fiscal year: 2000; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 2%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 8%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 69%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 19%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 2%. Fiscal year: 1990; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 25%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 30%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 39%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 4%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 2%. Weighted by population: Fiscal year: 2000; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 2%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 4%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 88%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 6%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 1%. Fiscal year: 1990; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 5%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 10%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 73%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 12%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 1%. Metropolitan areas: Fiscal year: 2000; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 3%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 6%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 86%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 4%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 1%. Fiscal year: 1990; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 8%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 14%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 71%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 8%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 0%. Nonmetropolitan areas: Fiscal year: 2000; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 2%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 8%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 66%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 21%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 3%. Fiscal year: 1990; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 29%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 31%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 34%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 4%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 2%. Sources: GAO analysis of HUD data (2000 figures) and HUD (1990 figures). [End of table] As table 1 shows, our analysis indicates that in 2000, where FMR estimates were higher or lower than the census by 10 percent or more, most often the FMR was too low, a different result from 1990 when HUD reported that most FMR estimates outside of the most accurate range were too high. Since the 2000 Census, HUD and others surveyed a limited number of FMR areas (153, as of September 2004). When we compared the rents derived from these surveys with FMR estimates in effect for these years, the outcome was similar to the results we found in our comparison with the 2000 census--almost three-fourths (73 percent) of FMR estimates were within 10 percent of the survey rents. When analyzing the 153 areas, we also found a difference in the results shown by rebenchmarking surveys undertaken for different reasons. HUD and PHAs conducted rebenchmarking surveys for two basic reasons: (1) HUD was adhering to a schedule in which it surveyed selected large metropolitan areas on a rotational basis or (2) HUD, PHAs, or others received information suggesting FMRs were inaccurate (usually a complaint that an FMR was too low) in a specific area. As shown in table 2, complaint-driven surveys (RDD surveys that were conducted by HUD following a PHA request or by a PHA itself) more often found inaccurate FMRs (i.e., FMR estimates that were 10 percent or more different from the rents derived from the survey). Table 2: Accuracy of HUD's FMR Estimates Compared with Rents from RDD Surveys (by Reason for Survey, 2001-05): Reason for survey: HUD schedule; Compared with RDD survey rents--percentage of FMRs that were: Higher by 20% or more: 0%; Compared with RDD survey rents--percentage of FMRs that were: Higher by 10% to 19.9%: 3%; Compared with RDD survey rents--percentage of FMRs that were: Within 10%: 87%; Compared with RDD survey rents--percentage of FMRs that were: Lower by 10% to 19.9%: 9%; Compared with RDD survey rents--percentage of FMRs that were: Lower by 20% or more: 1%. Reason for survey: By request (HUD surveyed); Compared with RDD survey rents--percentage of FMRs that were: Higher by 20% or more: 2%; Compared with RDD survey rents--percentage of FMRs that were: Higher by 10% to 19.9%: 2%; Compared with RDD survey rents--percentage of FMRs that were: Within 10%: 66%; Compared with RDD survey rents--percentage of FMRs that were: Lower by 10% to 19.9%: 26%; Compared with RDD survey rents--percentage of FMRs that were: Lower by 20% or more: 5%. Reason for survey: PHA surveyed; Compared with RDD survey rents--percentage of FMRs that were: Higher by 20% or more: 0%; Compared with RDD survey rents--percentage of FMRs that were: Higher by 10% to 19.9%: 0%; Compared with RDD survey rents--percentage of FMRs that were: Within 10%: 53%; Compared with RDD survey rents--percentage of FMRs that were: Lower by 10% to 19.9%: 42%; Compared with RDD survey rents--percentage of FMRs that were: Lower by 20% or more: 5%. Source: GAO analysis of HUD data. Note: HUD estimated FMRs we used in this comparison prior to the public comment step that takes place after its estimation process. When HUD received the results of RDD surveys prior to the public comment step, it used (and published) those rent estimates rather than the initial FMR estimate it had developed. As a result, some of the estimates we use in this comparison were never published by HUD as proposed FMRs. [End of table] According to HUD, those areas it surveyed because they were on its schedule were, like the complaint-driven RDD surveys, not random selections. Most often, HUD selected areas from its schedule because it had not surveyed them recently, which means that HUD tended to choose areas for which the length of time since the last rebenchmarking survey was longer. According to HUD officials, choosing areas for RDD surveys for this reason increases the likelihood that these surveys would find inaccurate FMRs. Further, to the extent that complaints are more likely to arise when FMRs are believed to be too low, rather than too high, it is not surprising that complaint-driven surveys were much more likely to show rents higher than FMRs, rather than lower. Quality Survey Data Tended to Produce FMR Estimates That Were Accurate within 10 Percent: Survey data that had one or all of the characteristics we summarize as "quality"--recent, accurate, or relevant--tended to more often produce FMRs within 10 percent of another rebenchmarking survey. Specifically, our analysis showed that FMR estimates more often fell in the most accurate range when HUD based FMRs on survey data that were (1) more recent, (2) taken from a higher quality survey than some surveys HUD has used in the past, or (3) more relevant because their source closely matches the boundaries of the FMR area. More Recent Data: FMR estimates that HUD rebenchmarked with newer survey data (1 to 4 years old) were associated with greater accuracy in 2000 (see table 3). For example, our analysis found that 88 percent of all FMR estimates based on newer data (i.e., 1 to 4 years old) were within 10 percent of the census estimates in 2000. Table 3: Accuracy of FMR Estimates in 2000 Compared with Rents from Census (Based on Age of Baseline FMR Data): Age of baseline FMR data: 1 to 4 years; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 1%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 6%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 88%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 5%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 0%. Age of baseline FMR data: 5 to 7 years; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 3%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 9%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 67%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 20%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 1%. Age of baseline FMR data: No survey from 1990 to 2000; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 2%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 7%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 67%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 20%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 3%. Source: GAO analysis of HUD data. Note: Areas based on 2000 decennial census data or 8-, 9-, or 10-year- old non-Census data comprised too few areas from which to calculate separate statistics. [End of table] In considering the association we found between recent data and accuracy, HUD officials stated that the length of time since the last rebenchmarking survey likely affected the accuracy of FMR estimates. As our analysis showed, areas for which the baseline data were older (including those for which there was no rebenchmarking survey between the 1990 and 2000 censuses) more often had FMR estimates that were 10 percent or more higher or lower than the estimate from a recent survey. Data from Higher Quality Surveys: When HUD used data from higher quality surveys than some surveys it had used in the past, its FMR estimates were accurate more often than when it relied on lesser-quality means, such as the traditional surveys some PHAs conducted before HUD adopted the RDD survey methodology. Currently, HUD uses the AHS or RDD surveys to rebenchmark FMRs between the decennial censuses. However, until the mid-1990s, HUD also, on occasion, accepted from PHAs and used for rebenchmarking FMRs survey data that PHAs collected via less rigorous traditional or telephone surveys.[Footnote 22] The AHS and RDD surveys can be considered higher quality than the less rigorous ones HUD once accepted because they have (1) survey characteristics required by HUD's regulations and guidelines and (2) data from a survey closely corresponding to the boundaries of the FMR areas. As shown in table 4, the higher quality survey sources- -AHS and RDD surveys--more often led to FMRs within 10 percent accuracy than the estimates based on less rigorous methods. Table 4: Accuracy of FMR Estimates in 2000 Compared with Rents from Census (Based on Type of Rebenchmarking Survey): Type of last FMR rebenchmarking survey: AHS; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 5%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 0%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 95%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 0%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 0%. Type of last FMR rebenchmarking survey: RDD-HUD; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 2%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 7%; Compared with decennial census rents- -percentage of FMRs that were: Within 10%: 86%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 5%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 0%. Type of last FMR rebenchmarking survey: RDD-PHA; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 3%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 22%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 72%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 4%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 0%. Type of last FMR rebenchmarking survey: Traditional; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 2%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 6%; Compared with decennial census rents- -percentage of FMRs that were: Within 10%: 65%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 26%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 1%. Type of last FMR rebenchmarking survey: Telephone; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 11%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 28%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 61%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 0%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 0%. Type of last FMR rebenchmarking survey: No Survey from 1990 to 2000; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 2%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 7%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 67%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 20%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 3%. Source: GAO analysis of HUD data. [End of table] More Relevant (Local) Data: When HUD used more relevant (local) surveys to update FMRs--that is, to adjust for inflation rather than to rebenchmark or revise the baseline- -the results were similar: FMR estimates were associated with greater accuracy. As shown in table 5, when HUD updated FMR estimates with the more local metro-specific CPI--a survey that generally matches the boundaries of metropolitan FMR areas--91 percent of estimates were within 10 percent accuracy. When HUD used regional RDD surveys--which cover much broader areas than the FMR area boundaries--to update FMR estimates, many fewer were within 10 percent accuracy. Table 5: Accuracy of FMR Estimates in 2000 Compared with Rents from Census (by Type of Update Factor): Type of update factor: Metro-specific CPI; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 5%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 1%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 91%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 3%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 0%. Type of update factor: RDD regional gross rent change factor; Compared with decennial census rents--percentage of FMRs that were: Higher by 20% or more: 2%; Compared with decennial census rents--percentage of FMRs that were: Higher by 10% to 19.9%: 8%; Compared with decennial census rents--percentage of FMRs that were: Within 10%: 68%; Compared with decennial census rents--percentage of FMRs that were: Lower by 10% to 19.9%: 19%; Compared with decennial census rents--percentage of FMRs that were: Lower by 20% or more: 3%. Source: GAO analysis of HUD data. [End of table] According to HUD officials, the use of broad factors--that is, factors from surveys covering a larger geographic area than the FMR area--for updating and trending in the FMR estimation process contributes to inaccuracy in the estimates. For instance, the update factors derived from regional RDD surveys may not capture changes in the local economy within a specific FMR area, such as a large employer leaving town or a sizable increase in the housing supply that may affect rents. Furthermore, HUD officials stated that the use of a nationwide factor for trending FMR estimates--the process of projecting FMR estimates into the future year for which they will be effective--may not capture local trends. (As previously noted, HUD currently applies to all FMR areas a standard trending factor derived from the change in the national average rents between the 1990 and 2000 censuses.) HUD Believes That Other Factors May Influence the Accuracy of FMR Estimates: In addition to the factors we identified as being associated with the accuracy of FMR estimates, HUD officials indicated several more factors that might also affect accuracy. Specifically, these officials cited (1) general survey error common to all such estimates, (2) the characteristics of nonmetropolitan areas, (3) difficulty in estimating utility costs, and (4) recent mover rent changes differing from rent changes captured by the CPI. General Survey Error: The data from the survey sources HUD uses are estimates which, by definition, can introduce error into FMR estimates. All surveys are subject to various types of error, which means that survey data may not precisely match the true value the survey is trying to measure. For example, sampling error occurs because a sample rather than an entire population was surveyed, and, according to HUD officials, census data for FMR estimates are generally subject to a 1 percent sampling error (in metropolitan areas). While HUD considers census data to be the best source for rent estimates (primarily because these data have a far larger sample size than any other source used), even the census includes some areas with low sample sizes or low response rates. Characteristics of Nonmetropolitan Areas: Our analysis showed that FMR estimates for nonmetropolitan areas were less likely to be based on quality data (more recent, taken from a higher quality survey and more relevant) and were also less likely to be more accurate. HUD officials told us that nonmetropolitan areas are a lower priority for rebenchmarking surveys between the censuses because they believe it is better to focus their limited resources (for their own rebenchmarking RDD surveys) on the areas where more potential voucher holders live (i.e., the metropolitan areas). Nonmetropolitan areas were less likely to have a recent rebenchmarking survey (sponsored by HUD)--between 1990 and 2000, HUD rebenchmarked 73 percent of all metropolitan areas and 31 percent of all nonmetropolitan areas. Also, HUD updates almost all nonmetropolitan areas using the broad update factors it derives from its regional RDD surveys, meaning that these areas' FMR estimates are updated with data that are less "local" than what HUD applies to the larger metropolitan areas with local CPI rent change estimates. Additionally, surveys of nonmetropolitan areas (even the census) often have relatively lower sample sizes than metropolitan areas, affecting the quality of the data for rebenchmarking FMR estimates there and, as a result, the accuracy of these estimates.[Footnote 23] Difficulty in Estimating Utility Costs: According to HUD officials, utility cost data are a source of error in all three survey data sources HUD uses to estimate FMRs. For example, renters have been documented as unreliable sources of the utility costs they pay, yet the census relies on them to report utility cost estimates. Utility costs for RDD surveys come from a utility cost schedule supplied by the local PHA; however, according to HUD officials, although PHAs certify that the data are correct, utility schedules can be unreliable and introduce bias into FMR estimates.[Footnote 24] The AHS uses a utility estimation model (consisting of certain survey variables) that HUD officials believe corrects to some extent for the error introduced by relying on tenant reporting. Nonetheless, they noted that the AHS model is based on survey estimates and thus remains subject to error in ways common to all surveys. Recent Mover Rent Changes in Metropolitan Areas: HUD officials told us that the local survey HUD uses for updating FMRs in some metropolitan areas--the metro-specific CPI--may not capture sudden changes in rents for recent movers. According to HUD, CPIs measure overall rent changes for all renters in a fixed group of units. However, rent changes for recent movers can be significantly different from changes for all renters. For example, HUD officials stated that San Francisco and Boston are among the more volatile housing markets in the country and, as a result, among the most difficult for which to estimate FMRs. Specifically, in 2000 and 2001, San Francisco's recent mover rents increased sharply, then decreased suddenly in 2002. However, the CPI for San Francisco, which covers all renters, showed above-average but not exceptional rent increases in 2000 and 2001 and no change for 2002. ACS Could Improve the Accuracy of FMRs by Providing HUD with More Recent, Better Data: The ACS, which is replacing the decennial census long form, could improve the accuracy of FMRs because it is a higher quality survey (compared with others HUD has available between the decennial censuses) and it provides more recent data that closely matches the boundaries of HUD's FMR areas. HUD plans to begin to use ACS data for fiscal year 2006 FMRs. However, certain challenges that we and others, including the National Academy of Sciences (NAS), have identified may affect the extent to which HUD can use ACS data to improve its estimates. County- level ACS data, which will be available each year, could increase the accuracy of FMRs because HUD plans to use them to rebenchmark all areas more frequently. Because the ACS data are more recent than the decennial census data and generally of similar quality and content, HUD plans to use ACS data to rebenchmark FMRs in generally the same way that it used the decennial census data in the past, but it will be able do so more frequently. Certain challenges for HUD regarding the ways ACS data are processed and reported may affect its plans for using them. For example, the Census Bureau averages ACS data over 1-, 3-, and 5-year time periods, and averaging could mask sharp trends in rents because it can smooth changes that occur within the time period. HUD plans to address these challenges after it receives fiscal year 2005 ACS data--the data collected during the first year of full ACS implementation--in Fall 2006. Despite the challenges in using these data, neither we nor experts and researchers who routinely work with housing data sources identified viable alternatives to the ACS. The ACS Is a Higher Quality Survey That Provides More Recent and Local Data: The ACS could improve the accuracy of FMRs because it is a higher quality survey than HUD currently has available between the decennial censuses and it provides more recent data closely matching the boundaries of HUD's FMR areas. Higher Quality Survey: The ACS is of higher quality than data sources (RDD surveys and the AHS) currently available to estimate FMRs between the decennial censuses. According to the Census Bureau, like its long-form predecessor, the ACS is the highest quality household survey currently conducted by the Census Bureau, and it will provide data more frequently.[Footnote 25] The ACS derives similar information as the decennial census long form, and its results undergo stringent processing by the Census Bureau. Moreover, according to HUD officials, the ACS is an impressive improvement over data from any other source. For instance, although the AHS is also a Census Bureau product, it is similar to RDD surveys because it provides data for only a comparatively small number of areas and does so less frequently. More specifically, the AHS covers a limited number of the largest metropolitan areas every few years. More Recent Data: Using ACS data to estimate FMRs could improve their accuracy because it provides HUD with more recent county-level data. More specifically, the ACS will provide data each year that is based on 1-, 3-, or 5-year rolling averages (i.e., the Census Bureau will collect data monthly, average them over 12 months, and publish new 1-, 3-, and 5-year averages each year). Because our analysis indicated that FMRs estimated with recent data (i.e., data that are 4 years old or less) more often tended to be within 10 percent of the results of a rebenchmarking survey, FMRs estimated with annual and 3-year average data could be more accurate. Even though FMRs estimated with 5-year average data would be based on some data older than 4 years, they could also be more accurate than is now the case because HUD could rebenchmark them every 5 years (as opposed to the 10 years between censuses). More Local Data: The ACS also will provide more local data--more specifically, state- level data--that HUD could use to update FMRs and therefore lead to more accurate FMRs. Currently, HUD updates FMRs for the majority of areas with regional RDD surveys, each of which provides HUD with aggregate gross rent change estimates based on data from up to eight states. As previously noted, our analysis suggested that when HUD estimated FMRs with more local data (i.e., data from a survey that closely corresponds to the boundaries of the FMR areas) more FMRs fell within the most accurate range. As a result, annual state-level ACS data could enable HUD to more accurately update FMRs. Although the state-level data do not closely correspond to the boundaries of FMR areas, they cover areas much smaller than the currently used RDD surveys. HUD Expects to First Use ACS Data to Update Fiscal Year 2006 FMRs: HUD expects to first use ACS data to update its estimated baseline rents when preparing fiscal year 2006 FMRs. To do so, HUD plans to use regional-level ACS data, rather than the more local state-level ACS data that will be available to it. The state-level ACS data would provide reliable data for geographic levels smaller than the areas covered by the regional ACS (or regional RDD surveys). However, according to HUD officials, they believe they need to obtain and work with the ACS data, assuring themselves of its reliability and usefulness before they will consider updating FMRs with the state-level ACS data. The effect of ACS data on FMR accuracy could be most notable once HUD begins to rebenchmark--not just update--FMRs with these data, which will likely begin with the fiscal year 2008 FMRs. HUD will use the first data available under ACS full implementation in Fall 2006 to rebenchmark fiscal year 2008 FMRs and plans to use them in ways similar to how it had used decennial census data because their content and quality are similar to that of the decennial census data. Figure 5 describes how often HUD could rebenchmark different-sized areas with ACS data, showing that, for example, HUD will likely rebenchmark FMRs for large metropolitan areas--where the most potential voucher holders live--every year. Figure 5: Scope of ACS Rebenchmarking as Related to FMR Area Size and Housing Choice Voucher Program Data: [See PDF for image] Note: The most recent available data for population and number of housing choice vouchers per FMR area are from fiscal years 2000 and 2003, respectively. We estimated the "voucher dollar" to approximate the relative dollar amounts of housing choice vouchers in each area. To do so, we multiplied the FMR (FY 2004) and the number of vouchers for each FMR area over 12 months. [End of figure] Because data developed from a single year of ACS data will be based on samples that are approximately one-sixth as large as decennial census long-form samples, HUD may need more data points than what the ACS will provide for communities with smaller populations in order to estimate FMRs. More specifically, according to HUD officials, to obtain a sufficient sample of rent data for HUD's program purposes, the agency needs data from areas with larger populations--that is, areas that can provide more data points--than the ACS will publicly report. For instance, in an annual ACS sample from a metropolitan area with a population of 100,000, HUD could expect to find in ACS data only 48 recent movers in two-bedroom rental units, but it needs 200 recent movers for its purposes. In order for HUD to obtain its needed minimum sample of 200 units, it will likely need to use 1-year average data for counties with populations of more than 400,000; 3-year average county- level data for areas with populations of 133,000 to 400,000; and 5-year average county-level data for areas with populations of less than 133,000. In addition, although the Census Bureau will publish 3-and 5-year rolling average ACS data every year beginning in 2008 and 2011, respectively, HUD may not use these data every year because of concerns about their reliability for HUD's FMR estimation purposes. According to the Census Bureau, reliable measures of changes in multiyear averages- -such as what HUD needs in order to estimate FMRs--should only be calculated using averages with no overlapping years. The 3-and 5-year rolling average ACS data that the Census Bureau publishes every year will have overlapping years. For example, in 2008, the Census Bureau will publish 3-year average ACS data covering 2005, 2006, and 2007; in 2009, it will publish 3-year average ACS data for 2006, 2007, and 2008, overlapping the previous year's estimate by including 2006 and 2007 data. For HUD's purposes, a reliable time series of 3-year averages would consist of the ACS data that the Census Bureau will publish in 2008 (2005-07 averages), 2011 (2008-10 averages), 2014 (2011-13 averages), and so on because these would not have overlapping years. ACS Data Pose Certain Challenges to HUD That May Affect FMR Estimation and Accuracy: HUD's consultant ORC Macro, NAS, the Census Bureau, and we have identified certain challenges associated with using ACS data that may affect how and when HUD could use the data and improve the accuracy of FMRs. The challenges include issues related to the averaging of the ACS data, presentation of inflation-adjusted costs (such as rents), techniques to deal with missing responses, and reporting differences between the decennial census and the ACS. Averaging: The Census Bureau collects data for the ACS monthly and continuously averages them over 1-, 3-, and 5-year time periods. However, this averaging could hide rental market shifts because moving averages tend to "smooth" changes in data over time.[Footnote 26] For example, if from January through September of a given year the rent for an area is $800, and from October through December of the same year the rent is $1,200, the average annual rent reported by the ACS would be $900, which is far less than the current monthly rent of $1,200. As a result, the moving averages' "smoothing" effect may hide a turning point, or, current prices in the rental housing market. Inflation-Adjusted Costs: To adjust for general inflation, the Census Bureau will use a general adjustment factor rather than an index that is specifically related to data items, such as rents or utilities, to present dollar-denominated data from the ACS. This could limit the usefulness of the data for HUD's program purposes because using a general adjustment factor (i.e., national CPI) rather than using an index that is specifically related to the dollar-denominated item (i.e., a rent index) could result in a less-precise estimate.[Footnote 27] The treatment of dollar- denominated data is critical to all users of these data, and particularly to HUD, which will be using the ACS to determine FMRs based on rent data. If HUD had access to the Census Bureau's unadjusted annual data, it could then adjust the data pertinent to its FMR estimation using rent or utility indexes. We previously raised concerns about the Census Bureau inflation adjustment.[Footnote 28] In response, the Census Bureau did not provide a rationale for using the general adjustment factor, rather than a more specific index, but did indicate that the bureau would reconsider its present policy of showing only the inflation-adjusted annual estimates. Techniques to Deal with Missing Responses: A NAS panel and we have raised concerns about how imputation--a technique used to deal with surveys with missing responses--could affect the accuracy of ACS data, especially in smaller areas. The NAS panel that reviewed the 2000 Census raised issues about the potential effects of imputation on ACS results. Unlike the process used for the decennial census--100 percent follow-up for all nonrespondents--the Census Bureau conducts follow-up on only 33 percent of nonrespondents to the ACS. The Census Bureau uses the responses from the follow-up surveys to attribute a similar pattern of responses to the remaining 66 percent of nonrespondents. The NAS panel called on the Census Bureau to analyze the associated trade-offs in costs and accuracy between imputation and additional fieldwork to gather more data.[Footnote 29] Reporting Differences between the Decennial Census Long Form and the ACS: In a 2004 study, the Census Bureau found that when the decennial census long form and the ACS were used to survey the same area, they reported a number of variables differently, including those HUD uses to estimate FMRs.[Footnote 30] The variables they reported differently include housing occupancy, the year the structure was built, the number of rooms, and gross rent. For instance, the study found that for certain areas, the ACS reported moderately lower gross rents than did the decennial census. According to the Census Bureau, the differences may result partly from different survey processing techniques or from the multiyear aspect of ACS data. Regardless of the cause, FMRs for fiscal year 2008 (the first year of rebenchmarking with ACS data) could show bigger changes than would be the case using decennial census data. According to HUD, consistent FMRs--that is, estimates that change gradually from year to year--are important because wide year-to-year fluctuations, especially those changes that lower the FMR, can be disruptive to PHAs, which must annually reconsider their payment standards any time HUD changes the FMR. HUD will address the ACS challenges when it receives and begins to analyze 2005 ACS data--that is, the data collected during the first year when the ACS is fully implemented--in Fall 2006. HUD may choose to participate in an ACS Technical Workshop led by the Census Bureau, which may help the agency address these challenges. Despite Challenges, the ACS Remains Likely the Best Data Source for FMRs: Despite the challenges the ACS poses for HUD, neither we nor various researchers and industry experts found reason to suggest (1) that HUD should not go forward with its plans to use the ACS or (2) that there are viable alternatives to the ACS. Other sources of information, such as private-market rent data and tax assessment data, typically do not contain the information that HUD needs to estimate FMRs. For example: * Private-market rent data typically include more expensive properties (i.e., luxury units, usually large apartment complexes, in metropolitan areas). Most voucher holders do not rent such properties because they cannot afford them. Additionally, these data do not include single- family homes--properties that voucher holders may also lease. * Private-market and tax assessment data are typically of lesser quality compared with the data sources that HUD generally uses to estimate FMRs. Private-market rent data often do not contain a representative sample of the full rent distribution in an FMR area. * Private-market or tax assessment surveys that include rent data may not consistently include questions that ensure the units included adhere to HUD's criteria (e.g., rents only from recent movers). HUD Did Not Follow One of Its Data Quality Guidelines and May Lack Data Sources to Assess the Accuracy of Future FMRs: The potential exists for HUD to improve how it estimates FMRs and their accuracy because (1) the agency presently does not follow its objectivity guideline for ensuring the transparency and reproducibility of its data and methods for estimating its FMRs and (2) it may in the future lack a way to assess the accuracy of ACS-based rent estimates when other information, such as comments from public housing agencies, suggests it may need to do so. Various federal agencies, including HUD, have developed guidelines to ensure they disseminate quality data. Three of HUD's standards--utility, integrity, and objectivity--apply to FMR estimation. Although HUD appears to be following the utility and integrity guidelines, it did not follow its objectivity guideline-- which calls for the agency to make its data sources and methods transparent so the results can be independently reproduced. Additionally, as HUD comes to depend less on RDD survey and AHS data, it may not have a means to assess the accuracy of future FMR estimates. HUD Has Not Followed Its Data Quality Guideline on Objectivity: Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Pub. L. No. 106-554) directs OMB to issue governmentwide guidelines that provide policy and procedural guidance to federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information disseminated by the agencies. According to OMB, information that has been subject to independent reanalysis is generally presumed to be of acceptable objectivity and therefore reliable to the user. In addition, OMB states that an important benefit of transparency and reproducibility (objectivity) is that the public can assess how much an agency's information hinges on the specific analytical choices of the agency. In response to OMB's guidelines, various federal agencies, including HUD, have developed similar guidelines for ensuring that they disseminate quality information. HUD's guidelines include ensuring the utility (usefulness), integrity (protection from unauthorized access), and objectivity (transparency and reproducibility) of the data it disseminates. Based on our review of available information, HUD appears to be following the utility and integrity components of its guidelines for FMRs. HUD's utility guideline states that the information disseminated should be useful--a standard that encompasses accessibility and timeliness. HUD follows this guideline by estimating FMRs on an annual schedule and making FMRs public and easily accessible by publishing them on its Web site and in the Federal Register. HUD's integrity guideline states that the information disseminated should be protected from corruption or falsification by unauthorized access or revision. According to HUD officials, FMR data are kept on an internal server with highly restricted access. Furthermore, to ensure the security of the system, the officials said they maintain full electronic backups of all systems. However, we found that HUD does not follow its guideline pertaining to objectivity. HUD's guidelines state that it will make publicly available the sources, data, and methods used to develop the information it disseminates, and that results must be capable of being "substantially reproduced." This means that independent reanalysis of original or supporting data using the same methods should generate similar analytical results. Although HUD generally describes its overall methodology for estimating FMRs in publicly available documents, the agency has not documented its methodology in sufficient detail to permit the results to be independently reproduced. For example, although we obtained information on the data and methods HUD used to estimate FMRs for fiscal years 2000-05, HUD's process was not sufficiently documented to allow us to reproduce FMRs without contacting HUD staff to assist us in doing so. In part, this was because some of the data HUD used to estimate FMRs, such as utility cost data, no longer exist after the agency upgraded the software it uses to develop FMRs. Also, HUD did not document some of the key procedures, variables, and data it used in estimating FMRs, such as the source of benchmarking data (and its rationale for choosing each source in any given year).[Footnote 31] Sufficient documentation would have allowed outside parties to understand and assess how HUD developed any given FMR. For example, sufficient documentation would allow an outside party to determine (1) every decision HUD made (such as the FMR area definition or survey source), (2) the decision rules it applied in making that decision, and (3) the extent to which HUD consistently applied these rules. HUD's Declining Use of RDD Surveys and AHS Data May Limit Its Options for Assessing the Accuracy of Future FMRs: HUD officials state that they do not have a plan to assess the accuracy of FMRs after they start using ACS data to estimate them, in part because they believe they will no longer have a quality comparison point or data with which to do so. In the past, HUD assessed accuracy by comparing FMR estimates with the rents derived from a subsequent RDD survey, the AHS, or a decennial census. However, HUD plans to limit its future use of RDD surveys and the AHS because of their concerns about cost and quality. According to HUD officials, RDD surveys are very expensive (costing upwards of $20,000) and their reliability is decreasing. Currently, according to HUD officials, the agency has to start with a sample of 97,000 units to obtain a usable sample of 200 with which to estimate FMRs. Moreover, the response rate for RDD surveys is about 40 percent, compared with 90 percent for the ACS, and RDD surveys may have nonresponse bias (i.e., people who respond to surveys may answer questions differently than those who do not). Similarly, the AHS is becoming less useful for HUD's purposes than when that survey first began. According to HUD officials, the number and sample sizes of AHS metropolitan area surveys has been decreasing over the past two decades, and they are not timely for HUD's program purposes, thereby making them less useful for estimating FMRs than has been the case in the past. Rent data from other sources, such as private-market rent surveys and tax assessment records, also would not provide HUD with a usable comparison point with which to assess FMR accuracy.[Footnote 32] Nonetheless, HUD's regulations require that the agency allow the public to provide comments on proposed FMRs, and its information quality guidelines permit affected parties to seek and obtain correction of information disseminated by the agency. This extends to the accuracy of FMRs. In addition to what its policies may require, even though FMRs based on ACS data will most likely be more accurate than previous FMRs, HUD officials acknowledge that ACS-based FMR estimates may be inaccurate from time to time. For example, FMRs for the smaller areas (rebenchmarked every 3 or 5 years with ACS data) may need to be assessed within the interval to ensure that they remain accurate between rebenchmarkings. Moreover, FMRs for any areas with volatile rental markets may need to be assessed with some frequency to ensure that they are accurate. However, as previously noted, HUD may lack sources of comparable data in the future and may be unable to perform these assessments. Conclusions: HUD's task in accurately estimating FMRs is formidable. It must produce estimates for hundreds of areas throughout the country despite having few comprehensive, reliable data sources with which to do so. Additionally, HUD faces the normal difficulties associated with predicting how rents and housing markets will change months (or years) into the future. Nonetheless, for those affected by FMRs, such as voucher holders, HUD's ability to produce accurate estimates each and every year is vital--estimates that are too low make it more difficult for low-income households to find housing they can rent with a voucher, while estimates that are too high may needlessly waste resources or prevent housing agencies from serving more households. At the time of our review, HUD could not dispel concerns about its process for estimating FMRs because its methodology is not transparent enough to allow others--including GAO--to independently analyze its rent data and produce similar results. HUD and those who use FMRs would benefit from a more transparent methodology because this could enhance the credibility of the estimates by clearly delineating the choices HUD makes, what alternatives it may have had in making those choices, and the decision rules it applied; for example, whether to use OMB's area definitions or how much to modify them (and the basis for doing so). Making the methodology transparent would also give users more and better information with which to consider whether FMRs reliably reflect an accurate estimate of the rents voucher holders and others will encounter. The advent of a new data source holds promise for HUD because a system of FMRs that are largely based on the ACS will likely improve the quality and accuracy of these estimates. However, the level of improvement in the quality and accuracy of FMR estimates depends on how HUD uses the ACS data. By choosing to use regional-level data to update fiscal year 2006 FMRs rather than the more local state-level data, HUD may not be taking full advantage of the new data source as soon as it can. In addition, as it transitions to the ACS, HUD expects to discontinue its use of other surveys like the RDD surveys and the AHS to assess the accuracy of its FMRs and, therefore, will not have a means to assure itself and others that any given FMR estimate is accurate, particularly when it receives public comments or other information suggesting it needs to do so. While we agree that HUD is right to be concerned about the escalating costs and declining quality of surveys such as the RDD surveys, having no reasonable alternative to assess the accuracy of an FMR will not likely address the concerns of PHAs with reason to question FMR accuracy and may also contradict HUD's own data quality guidelines. Recommendations for Executive Action: To improve the usefulness of its FMR estimates, we recommend that the Secretary of HUD take the following three steps: * ensure that HUD fully documents its method for estimating FMRs by following all of its data dissemination quality guidelines, particularly those pertaining to the transparency and reproducibility of its methodology; * use, as much as possible, the ACS data that corresponds more closely to FMR areas to update the fiscal year 2006 FMRs; and: * develop a mechanism to assess the accuracy of future FMRs, including those that are based on the ACS, in instances where HUD learns of information suggesting it needs to do so. Agency Comments and Our Evaluation: We provided a draft of this report to HUD for its review and comment. In a letter from the Assistant Secretary for Policy Development and Research (see app. II), HUD described our report as a good summary of the intent of FMR estimates and the implementation of its methods. HUD also suggested certain changes and clarifications to our report. For example: * HUD suggested that we present population-weighted accuracy estimates in the "Highlights" page of our report. We agree that population- weighted estimates are important and note that we present the information in the body of the report rather than the "Highlights" page. * HUD provided a revised statement describing the process they use to eliminate subsidized and nonstandard housing units from the rent distribution. As HUD requested, we incorporated the new language in footnote 16. HUD agreed with our recommendation that it can better document its methods for estimating FMRs, but also requested that we clarify certain transparency and reproducibility issues in our report and recognize its ongoing efforts in this regard. Among other things: * HUD noted a distinction between process transparency and reproducibility of results, stating that the public's needs are better met by providing an overview of how FMRs are calculated and then showing the individual calculations for each FMR estimate, rather than providing system technical documentation, such as computer programs and input data. * HUD has sought to make the data and calculation process publicly available and transparent. For example, HUD noted that it currently posts on its Web site publicly releasable versions of 2000 decennial census detailed rent distribution files; FMR history files from the Federal Register, including Annual Adjustment Factors; and a summary of the general methodology and major data sources it uses to estimate FMRs. * HUD stated that it provided us with additional information, such as computer programs and input data, it used to estimate FMRs, and met with us as needed to explain the FMR methodology, including the large number of different data sources, decision rules, complex decision trees, and complex series of computer programs it uses to estimate FMRs. We agree that providing step-by-step calculation details for each FMR estimate would contribute to process transparency. Moreover, we agree that HUD currently makes the major data sources and general methodology it uses to estimate FMRs publicly available on its Web site. However, as our draft report noted, the current information that HUD makes publicly available does not show the individual calculations for each FMR estimate and therefore is not sufficient to substantially reproduce FMRs, a standard set out in HUD's data quality guidelines. With respect to reproducibility, in reviewing HUD's process for estimating FMRs, we asked for and HUD provided additional information, such as computer programs, input data, and associated documentation. Because HUD did not have and could not provide us with critical documents, such as a clear step-by-step guide or data dictionary, HUD officials met with us to explain the various computer programs and variables they used--a step that should not be necessary if the objective is for us to be able to independently substantially reproduce FMR estimates. Nonetheless, the information and explanations HUD provided were not sufficient to allow us to independently reproduce FMR estimates. As HUD noted in its comments, documentation of computer programs and input data, such as it provided us, are not as useful as step-by-step guidelines that clearly detail how it produces each FMR. As a result, HUD indicated it plans to consolidate in one place all of the information it uses to estimate FMRs and create a new tool, for release in April 2005, detailing how it develops each FMR. By making this information publicly available on its Web site, HUD expects to improve the transparency and reproducibility of its FMR estimates, particularly for the users of these estimates. HUD disagreed with our recommendation that it use, as much as possible, the ACS data that corresponds more close to FMR areas to update its fiscal year 2006 FMRs. HUD indicated that many of the annual state- level rent numbers have a pattern of erratic changes. However, according to the Census Bureau, for states with populations of 1 million or more, annual ACS changes for 2001 to 2004 are generally reliable. More importantly, as HUD officials indicated to us during our review, a necessary first step in using these data to update fiscal year 2006 FMRs would be to assess for each state whether anomalies or other concerns might indicate a need to defer in certain instances using the state-level ACS data. Accordingly, our recommendation was for HUD to use the state-level data as much as possible, recognizing that the agency could do so only in instances where the ACS data are sufficiently reliable for this purpose, and we have retained the recommendation. HUD did not explicitly agree or disagree with our recommendation that it develop a mechanism to assess the accuracy of future FMR estimates. However, HUD disagreed with our draft report's statement that declining use of RDD surveys and the AHS may limit its options for assessing FMR accuracy. Specifically, HUD stated that even though the ACS will be much more accurate than any other survey unless the other survey offers more current estimates, ACS rent estimates will always lag by at least a year (from the midpoint of the survey estimate); thus, use of national rent data to trend the FMR estimate could lead to estimation errors in housing markets with unusual rent increases or decreases. Accordingly, HUD noted that one of the major challenges posed by the ACS is how to identify those areas where the use of regional or national trending factors results in estimation inaccuracy, and stated that it is currently exploring two alternatives to deal with the issues. Thus, although HUD stated that it disagreed with our statement, the actions that it intends to take are consistent with our recommendation. HUD also suggested technical clarifications to our report, which we have incorporated as appropriate. As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to the appropriate congressional committees and to the Secretary of Housing and Urban Development. We will also make copies available to others upon request. In addition, this report will be available at no charge on the GAO Web site at [Hyperlink, http://www.gao.gov]. If you have any questions about this report, please contact me at (202) 512-6878 or [Hyperlink, woodd@gao.gov] or Bill MacBlane, Assistant Director, at (202) 512-6764 or [Hyperlink, macblanew@gao.gov]. Key contributors to this report are listed in appendix III. Sincerely yours, Signed by: David G. Wood: Director, Financial Markets and Community Investment: [End of section] Appendixes: Appendix I: Objectives, Scope, and Methodology: To describe how the Department of Housing and Urban Development (HUD) estimates fair market rents (FMR), we first analyzed statutes and HUD regulations, reviewed HUD documents, and interviewed HUD officials to identify each step that HUD takes to estimate FMRs, including the role that the public has in the process. Further, we also spoke with nine HUD field economists for each HUD region--typically, the first point of contact for the public--to further understand the role that the public can play in adjusting the FMR estimate.[Footnote 33] To identify and describe the relevant characteristics of the major data sources HUD uses to estimate FMRs, we reviewed agency documents. To determine how accurate FMRs were, we compared two-bedroom FMRs that HUD had in place for fiscal year 2000--that is, estimates derived from HUD's revisions to its baselines and from its update processes--with the results of the 2000 census.[Footnote 34] In addition, we compared two-bedroom FMRs that HUD estimated for fiscal years 2001-05 with data from surveys HUD and others conducted for 153 FMR areas over this period. We assessed accuracy by way of a comparison to the decennial census or other surveys because our own methodological experts as well as others conducting similar research on these issues determined that such a comparison is the best way to do so when the true values--that is, the distribution of all rents--cannot be known. In conducting both of these comparisons, we focused on two-bedroom units because HUD directly estimates FMRs for these units from the decennial census and its other rebenchmarking surveys. HUD does not directly estimate FMRs for other bedroom sizes, making it not possible to do a comparison of those FMRs to the results of a survey such as the American Housing Survey (AHS) or a Random Digit Dialing (RDD) survey.[Footnote 35] We performed an associative analysis to determine what components of HUD's FMR estimation process may have explained the results we found when we assessed accuracy (e.g., whether the estimate was for a metropolitan or nonmetropolitan area).[Footnote 36] Our analysis was limited to making associations between the components of HUD's methodology and the accuracy of its FMR estimates; it did not allow us to make a direct causal link between the two because all of the information we needed was either no longer available or may not be able to be captured by HUD's method for making these estimates. Specifically, (1) HUD could not provide all of the data used to estimate FMRs from 1990 to 2005, such as utility cost data, because these were kept on individual staff's computers and in many cases were not transferred when HUD moved its FMR data systems to a more advanced server; (2) the lack of transparency we found relative to HUD's objectivity guideline for data quality meant that we could not identify and isolate specific components of its methodology to attempt a causal (rather than associative) analysis; and (3) neither we nor HUD can control for factors outside of HUD's estimation process that may affect accuracy, such as sudden employment changes that cause an area's rents to increase rapidly. We present our analysis of the accuracy of FMR estimates in terms of the degree (percentage) to which the FMR matched or was close to the corresponding survey. For example, for the corresponding fiscal year 2000 FMRs and census data, we calculated the following for each FMR: Survey (census)-Fair Market Rent Estimate = x percent Survey (census): This calculation produced a percentage that, in this example, we characterize as the estimate being within x percent of the census. For descriptive purposes, we arrayed these comparisons in increments of 10 percent because, in terms of the initial FMR, this is the range (90 to 110 percent of the FMR) in which the public housing agencies may set their payment standards without prior approval from HUD. To determine how and when the incorporation of the American Community Survey (ACS) data might affect the accuracy of the FMR estimates, we reviewed agency documents and interviewed HUD officials to determine how the agency plans to use ACS data to estimate FMRs. We also analyzed Bureau of the Census documents to compare characteristics of the ACS data with those of the data sources HUD currently uses (the decennial census long form, the AHS, and RDD surveys) to estimate FMRs. Additionally, we reviewed research by the National Academy of Sciences and ORC Macro, in addition to our own, on the use of ACS data. To identify changes HUD could make to improve the way it estimates FMRs and their accuracy, we first assessed HUD's process for estimating FMRs against its data quality guidelines. More specifically, we analyzed each HUD guideline--utility, integrity, and objectivity--and compared them with HUD's method for estimating FMRs. We also interviewed HUD officials to determine how the guidelines related to FMRs. Additionally, on the basis of our analysis of the data characteristics we found to be associated with greater accuracy in FMRs (recent, higher quality, and more local), we interviewed housing industry experts that either routinely work with housing data or are familiar with HUD's data needs to identify potential alternative data sources that HUD could use to estimate FMRs. We also interviewed HUD officials to determine the availability and merits of alternative data sources. We conducted our work in Washington, D.C., between May 2004 and February 2005 in accordance with generally accepted government auditing standards. [End of section] Appendix II: Comments from the Department of Housing and Urban Development: U.S. DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT: ASSISTANT SECRETARY FOR POLICY DEVELOPMENT AND RESEARCH: WASHINGTON, DC 20410-6000: March 18, 2005: Mr. David G. Wood: Director, Financial Markets and Community Investment: U.S. Government Accounting Office: 441 G Street, NW, Room 2440: Washington, DC 20548: Dear Mr. Wood: Thank you for the opportunity to comment on GAO's draft report on Section 8 Fair Market Rents, GAO-05-342. GAO's review of HUD's Section 8 Fair Market Rent (FMR) system provides a good summary of the intent and implementation of the system. There are some items, however, for which clarifications or changes are warranted. There are also some points with which we disagree for reasons noted. The major points where clarifications or changes are requested are as follows: 1. The title of the "Highlights" report is inconsistent with the content. The current title suggests that FMR estimates are inaccurate but, as noted below and in the body of GAO's report, this is not the case. A more accurate summary of the report would be that "HUD FMRs are Generally Accurate And Should Improve With ACS Data, But Calculations Should Be Made More Transparent." 2. The unweighted FMR area accuracy measure used in the "Highlights" section is misleading. The accuracy percentages reported treat all 356 metropolitan and 2,303 non-metropolitan areas as equals. Loving County, Texas, with a population of 67 is treated as the equal of Los Angeles with its 9.5 million people. The most relevant accuracy standard would be a voucher-weighted percentage, followed by a population-weighted measure. Population-weighted accuracy percentages are provided in the body of the paper, but the initial and misleading impression in the "Highlights" section will receive more attention. Attachment 1 provides data on all three measures. The voucher-weighted accuracy results of 91 percent overall and 95 percent for metro areas should be referenced in the summary. It is requested that the second paragraph of the "Highlights" section be replaced as follows: "The 2000 Census rent data needed to measure the accuracy of FMRs became available four years after the Fiscal Year 2000 FMRs were published, and provide the most reliable available accuracy reference standard. Ninety-one percent of all Section 8 vouchers in 2000 were in areas with FMRs that were within 10 percent of FMRs calculated using the 2000 decennial Census. Ninety-five percent of metropolitan area vouchers were in areas that met this standard. An equivalent accuracy measurement is not available for 1990, but on a population-weighted basis 73 percent of all FMRs were within 10 percent of 1990 Census- based FMRs and 94 percent were within 20 percent. In 2000, 88 percent of all population-weighted FMRs were within 10 percent of 2000 Census- based FMRs and 98 percent were within 20 percent. This is a significant improvement over FMR accuracy in 1990. Given that areas considered likely to have FMR accuracy problems are given first priority for FMR surveys, it is also indicative that about 73 percent of the 153 areas surveyed since the 2000 Census had published FMRs within 10 percent of the survey estimates and that many of the areas where concerns were raised were found to have accurate FMRs." 3. FMR transparency and reproducibility issues raised in GAO's stud require clarification and recognition of HUD efforts in these areas. There are two approaches that could be taken to making FMRs reproducible by interested members of the public. One is to provide all of the data and computer programs used plus a detailed explanation of system functioning and decision rules. There are statutory restrictions on release of some of these data, but HUD has had Census produce publicly releasable, close approximations of the data actually used. The major deficiency of a systems documentation-based approach is that, while any given FMR was developed with a limited number of calculations, there are a sufficiently large number of different data sources and decision rules to require complex decision trees and a complex series of computer programs. Although HUD is willing to provide all system documentation and computer programs, and did so for GAO's review, its experience with GAO confirms its belief that the public's needs are better met by providing an overview of how FMRs are calculated and then showing the individual calculations for each FMR area. HUD is in agreement that additional efforts to improve transparency and reproducibility are desirable, but requests that GAO's report be modified in recognition of the following: a. There is a significant difference between transparency and system reproducibility. HUD has increasingly sought to make the data and calculation process as publicly available and transparent as possible. Its approach has been to make as much of the underlying data as possible available and to provide a step-by-step guideline to show how local FMRs were produced. Doing this is much easier than trying to provide understandable versions of the computer programs and processing stream used to produce FMRs. Documenting the FMR system to meet OMB and HUD's Office of Information Technology documentation standards, as was done in 2002, was both expensive and staff-intensive. HUD agrees that this latter type of documentation is not useful to a typical interested party, and believes that providing step-by-step calculation details for each FMR area is more useful and provides more estimation transparency than trying to expand on system technical documentation. b. HUD has actively sought to make the FMR process more transparent and reproducible. Examples follow: i. HUD had the Census Bureau prepare special, publicly releasable versions of the detailed rent distribution files used to calculate FMRs and bedroom rent ratios as well as an explanation of the calculation process and factors used. These have been placed on the HUDUSER website with file documentation. ii. HUD provides FMR history files, proposed and final Federal Register FMR publications, FMR Annual Adjustment Factors, and a summary of the methodology and data sources on its HUDUSER website. iii. HUD has developed an easy-to-use, Fiscal Year 2005 FMR replication tool that permits users to select an area of interest and to see how the area's 2000 Census FMRs and bedroom ratios were developed. It tracks the year-by-year process to update 2000 Census estimates (or more current survey estimates) to produce revised final Fiscal Year 2005 FMRs. The information provided includes the source and type of all update factors used. Historical FMRs and related updating documentation will be provided by a related reference tool, although the underlying information is already posted on the HUDUSER website. The new reference tools are still being tested and improved, but should be made available to the public in April 2005. A test version is available for GAO use at: http://www.buduser.org/datasets/fmr/computations.btm. Recognition of these efforts in the GAO report is requested. c. HUD provided GAO with full documentation for Fiscal Year 2005 FMRs. This included all FMR calculation processes, all computer programs, all input data, and the step-by-step process that updated Census 2000 rents and produced Fiscal Year 2005 FMRs, including the decision rules for selecting each of the FMR data sources used. HUD also volunteered to answer any questions related to this process and spent many hours doing so with GAO staff. Except for Census 2000 base rents (which are based on embargoed data), Fiscal Year 2005 FMRs can be reproduced to the dollar by following the documentation provided to GAO. HUD also offered to show how any given FMR rebenchmarked since the 1990 Census was derived, but no longer maintains the mainframe and PC-based mix of computer systems needed to systematically replicate all estimates at once. d. HUD disagrees with the criticism on page _47 that its estimates are not capable of being "substantially reproduced". The benchmark FMRs for any given year are from the Census, the American Housing Survey, a Random Digit Dialing survey, or a local survey. The data sources and amounts for any rebenchmarked FMRs are specified in the Federal Register and in the FMR history file available on the HUDUSER website. The FMR Annual Adjustment Factors (AAFs) used to update FMRs each year are published in the Federal Re ig ster and posted on the HUDUSER website. FMR estimates for any area can be reproduced with these data, although the consolidation of all relevant data in one place, as being done in the system that will be shortly released to the public, will make reproducibility easier for the public. 4. Footnote 16 is misleading because it implies that the Fiscal Year 2005 FMRs introduced a proxy to eliminate subsidized and nonstandard units, whereas HUD has always had to use proxies. This topic is relatively complex and is one on which HUD is seeking outside expert review to provide additional perspectives. HUD uses the housing quality measures available in the Census to eliminate substandard units from the rent distributions used to calculate FMRs. Those measures are limited, and HUD's objective is to reflect a more rigorous quality standard equivalent to Section 8 housing quality standards. A statistically process for doing this based on extensive research is used that works well for most areas. Moreover, available data from various Census Bureau studies suggest that serious housing quality deficiencies affect a very small percentage of the inventory and have little impact on most FMR estimates. Inclusion of income-based, subsidized housing rent charges is a matter of more concern in developing FMR estimates. Inclusion of public housing and HUD Section 8 housing assistance programs could distort FMR estimates, and 2000 Census data do not identify such assisted housing units. HUD examined the adjustment approach used with the 1990 Census data to correct for this bias. It also examined alternatives that took advantage of ACS and HUD administrative data in conjunction with 2000 Census data, and implemented an approach that has the effect of ensuring that a larger number of rental units are removed from the bottom of the respective local rent distributions prior to estimating FMRs than the number of locally assisted housing units. It is requested that the original footnote 16 be replaced with the more accurate statement that follows: "Prior to 2005, HUD used data on unit quality and assistance from the American Housing Survey to generate a proxy for public, assisted and substandard housing. This adjustment was constant over the nation and did not vary by bedroom size. In the 2000 rebenchmarking, HUD employed American Community Survey and HUD administrative data to calculate a sub-standard housing adjustment that is tailored to region and bedroom sizes. This new proxy allows for larger adjustments in areas with more public and assisted housing units and higher housing quality issues. HUD still uses information from RDD and AHS surveys to eliminate subsidized and nonstandard units from survey data." 5. The discussion of the American Community Survey (ACS) is internally- inconsistent and is likely to confuse readers. Pages 35 and 36 of the GAO draft report state that the ACS is a higher quality survey than others available except the Census, that it offers more current data, and that "despite challenges in using the data, neither the GAO nor experts and researchers who routinely work with housing data sources identified viable alternatives to the ACS." Page 44 reiterates this conclusion. The report then goes on to raise concerns about the ACS and the lack of HUD efforts to develop alternatives to test the accuracy of the ACS. In response, the following comments are offered: a. GAO's concerns about the ACS accuracy apply almost equally to Census data-the sample sizes and response rates are so much higher than from any other source that evaluating the accuracy of the data using other reference information is rarely feasible. The ACS is so much better than other data sources that it will become the "gold" reference standard once full annual samples are available. The ACS should provide annual FMR estimates almost as reliable as those from the Census for the largest metro areas and provide highly reliable estimates for all other metro areas using either two or three years of data. HUD's concern with the wording of the GAO report is that uninformed readers may miss the point that the ACS is an impressive improvement over any data from any source other than the decennial Census. b. HUD disagrees with GAO's recommendation to use partial ACS sample data at the state rather than the HUD Regional level in developing Fiscal Year 2006 FMRs. The ACS sample was not fully implemented until the start of 2005, and the previous year samples were much smaller and highly clustered. In analyzing ACS data, we have found that many of the annual state-level rent numbers show a pattern of erratic change suspiciously similar to what one would expect from small samples. We have no supporting evidence that instances where unusually high rent changes are followed by unusually low rent changes (and vice versa) reflect reality rather than sampling variability, which is why we believe it is preferable to use multi-state regional rent change factors until full sample ACS data for 2005 and subsequent years start to become available. 6. HUD disagrees with the statement on page 48 that declining use of RDD surveys and AHS data may limit its options for assessing the accuracy of future FMRs. First, HUD anticipates continuing to review AHS surveys and to make limited use of RDD surveys. HUD's underlying disagreement with this statement, however, relates to the relative accuracy of the ACS surveys. The ACS surveys will be so much more accurate than AHS or RDD surveys for metropolitan areas that, as noted in the GAO report on page 44, other available surveys are less reliable and therefore cannot be used to evaluate the accuracy of fully implemented ACS estimates. Other surveys are likely to be of value only if they offer more current estimates. ACS survey data releases will always lag by at least a year from the mid-point of the survey estimate, and there will be longer lags for all but the largest areas. Use of national rent trend information to cover the period from the ACS survey to the as-of date of the FMR estimate will work well for the large majority of areas, but lead to estimation errors in housing markets with unusual rent increases or decreases. One of the major challenges posed by ACS data is how to identify the relatively small number of areas where the use of regional or national trending factors to cover the lag period results in estimation accuracy problems. HUD is currently exploring two alternatives to deal with this issue. Thank you for your consideration of these comments. Please do not hesitate to contact me at 202-708-1600 should you have any questions. Sincerely, Signed for: Dennis C. Shea: Attachment 1: Published Versus Subsequently Available Census-Based FMRs: [See PDF for image] [End of table] [End of section] Appendix III: GAO Contacts and Staff Acknowledgments: GAO Contacts: David G. Wood, (202) 512-6878; Bill MacBlane, (202) 512-6764: Staff Acknowledgments: In addition to the individuals named above, Triana Bash, Tania Calhoun, Steve Brown, John Larsen, Marc Molino, Chris Moriarity, Robert Parker, MacDonald Phillips, Carl Ramirez, Barbara Roesmann, and Anita Visser made key contributions to this report. (250205): FOOTNOTES [1] As part of the decennial census since 1960, the Census Bureau has mailed separate long-form questionnaires to a sample of households to collect detailed information on demographic, housing, social, and economic characteristics. [2] When households rent units for less than the payment standard, the HUD subsidy is the difference between their gross rent and their income contribution. [3] The Moderate Rehabilitation Single-Room Occupancy (SRO) program provides rental assistance for homeless persons in connection with the rehabilitation of SRO dwellings. [4] HUD's Mark-to-Market Program reduces rents to market levels for expiring housing subsidy contracts and restructures existing debt to levels supportable by these rents on thousands of privately owned multifamily properties with federally insured mortgages. [5] HUD's HOME Program helps to expand the supply of decent, affordable housing for low-and very-low-income families by providing grants to states and local governments to fund housing programs that meet local needs and priorities. [6] HOPWA addresses the specific needs of persons living with HIV/AIDS and their families by making grants to local communities, states, and nonprofit organizations for purposes such as facility operations or rental assistance. [7] The LIHTC Program is an indirect federal subsidy used to increase the supply of affordable housing in communities by financing the development of affordable rental housing for low-income households. Difficult development areas are designated by the Secretary of HUD as areas that have high construction, land, and utility costs relative to the area median gross income. [8] 42 U.S.C. § 1437f(c)(1). [9] See 24 C.F.R. § 888.113 for regulations governing the FMR methodology. [10] Beginning in 2001, HUD set FMRs for 39 metropolitan areas at the 50TH percentile, because it determined that an FMR increase was needed to promote residential choice, help families move closer to areas of job growth, and alleviate concentrations of poverty. [11] Rents for units on 10 or more acres and seasonal units, such as summer rentals, are ineligible for the FMR estimation process. [12] The ACS is subject to annual appropriations. Funding for the ACS to cover all persons except those living in group quarters (e.g., college dormitories and prisons) was approved beginning with fiscal year 2005. Funding to cover all persons has been requested beginning with 2006. [13] The first annual ACS data for geographic areas with populations larger than 65,000 will be published beginning in 2006; publication of 3-year averages for areas with populations of 20,000 to 65,000 will begin in 2008; and publication of 5-year averages for areas with less than 20,000 will begin in 2010. [14] According to OMB, a metropolitan area generally consists of a core area containing a substantial population nucleus, and adjacent communities exhibiting a high degree of economic and social integration with the core. [15] In 1994, we reported on a proposal to establish smaller FMR areas. See GAO, Rental Housing: Use of Smaller Market Areas to Set Rent Subsidy Levels Has Drawbacks, RCED-94-112 (Washington, D.C.: June 24, 1994). [16] Prior to 2005, HUD used information on unit quality and assistance from the AHS to generate a proxy for subsidized (public and assisted) and substandard housing. This adjustment was constant over the nation and did not vary by bedroom size. To estimate fiscal year 2005 FMRs, HUD used ACS and HUD administrative data to calculate a substandard housing adjustment that is tailored to region and bedroom sizes. Specifically, HUD began to use the 75TH percentile of public housing rents from its administrative data for each of its regions as a proxy to indicate which units are subsidized and substandard. According to HUD, this new proxy allows for larger adjustments in areas with more public and assisted housing units and higher housing quality issues. HUD continues to use information from RDD surveys and the AHS to eliminate subsidized and substandard units from survey data. [17] The 2000 decennial census produced data sufficient to allow HUD to directly estimate FMRs for all bedroom sizes for fiscal year 2005 FMRs and update the bedroom ratios. According to HUD officials, they will use these new ratios to estimate future non-two-bedroom FMRs. [18] In very limited instances, HUD officials will accept data from PHAs in areas with small populations that have not followed the requirements. According to HUD officials, some areas with small populations will not be able to comply due to limited budgets or small sample sizes within the FMR area. HUD officials then evaluate the data on the basis of their professional judgment. [19] In order to obtain an exception to increase the payment standard by more than 10 percent, the public must submit documentation that demonstrates approval of the special exception is necessary to prevent financial hardship for families in the exception area. This documentation can include census rent data, locally funded quality surveys, lease rates, and success rates. The request must be needed (1) to enable families to find housing outside areas of high poverty and (2) because voucher holders have trouble finding housing for lease. [20] We use "associated with accuracy" because our analysis does not enable us to make causal links between survey or FMR area characteristics and the accuracy of estimates. [21] Most households receiving tenant-based vouchers--85 percent--live in metropolitan areas. [22] Traditional surveys are surveys of rent data in metropolitan areas with relatively low populations in which a PHA or other entities have access to all or almost all of the rents in the area--for example, in cities or towns that require owners to register rents annually and maintain a database of rents. Telephone surveys are generally derived from randomly selected lists of residential telephone numbers but are not assisted by the use of a computer to track telephone calls and the outcomes. [23] According to HUD, some nonmetropolitan areas have unusually low census data sample sizes and unusually high levels of substandard and assisted housing that may distort the accuracy of FMR estimates. For the fiscal years 1996 through 2004 FMRs, HUD corrected for low FMR estimates that were at or below the cost of operating housing by implementing a minimum FMR level for each state. [24] According to HUD, a PHA utility schedule is a list of the average monthly costs of various types of utilities, such as heating oil, electricity, or water and sewer charges, subdivided by the number of bedrooms in the unit. [25] The Census Bureau reports that the 5-year averages will be about as accurate as the long-form data; the annual and 3-year averages will be significantly less reliable than the long-form data but more reliable than existing household surveys the Census Bureau conducts. [26] See ORC Macro, The American Community Survey: Challenges and Opportunities for HUD (Calverton, MD: Sept. 27, 2002). ORC Macro is the consultant HUD hired. [27] See GAO, American Community Survey: Key Unresolved Issues, GAO-05- 82 (Washington, D.C.: Oct. 8, 2004). [28] See GAO-05-82. [29] See GAO-05-82. [30] U.S. Census Bureau, Meeting 21ST Century Demographic Data Needs- Implementing the American Community Survey Report 8: Comparison of the American Community Survey Three-Year Averages and the Census Sample for a Sample of Counties and Tracts (Washington, D.C.: 2004). [31] For example, if the survey source for an FMR estimate was the AHS, HUD's documentation did not indicate what other sources, if any, it considered that year and why it chose the AHS over any other available sources of data for that year. [32] According to ORC Macro, HUD may be able to use special ACS tabulations from the Census Bureau to detect shifts in rent trends for areas where HUD will use multiyear average data to estimate FMRs. These data for each FMR area may not contain enough samples to estimate FMRs, but would give HUD an indication that an existing FMR may be inaccurate. [33] As of December 2004, nine HUD regional field economists managed the agency's economic work in the 10 HUD regions because there was a vacancy in Region 2 (New York/New Jersey). [34] In order to use the 2000 decennial census data we obtained from HUD to assess the accuracy of FMRs, we verified the reliability of the census data by asking HUD officials a series of data reliability questions. [35] For non-two-bedroom units in the 2000 decennial census survey and 153 subsequent rebenchmarking surveys, HUD took the survey results for two-bedroom rents and applied a rent ratio that, in HUD's view, captured the approximate relationship between rents for two-bedroom units and other sizes. For example, through fiscal year 2004, for three- bedroom units, HUD determined that the relationship between these and two-bedroom rents was 1.25, so the three-bedroom FMR would be 125 percent of what HUD estimated for two-bedroom units. [36] When we compared the accuracy of FMR estimates with the two types of update factors HUD uses (metro-specific Consumer Price Index or RDD regional gross rent change factor), we excluded a limited number of FMR areas because HUD applies special rules for updating this group, making the update calculations too dissimilar for our purposes. GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.