Rental Housing
HUD Can Improve Its Process for Estimating Fair Market Rents Gao ID: GAO-05-342 March 31, 2005The Department of Housing and Urban Development (HUD) annually estimates fair market rents (FMR) for standard quality rental units throughout the United States. Among other uses, FMRs help determine subsidies for almost 2 million low-income families in the nation's largest rental assistance program. However, concerns exist that FMRs can be inaccurate--often, too low, preventing program participants from finding affordable housing. Also, HUD will soon derive FMRs from a new source, the American Community Survey (ACS), which processes data somewhat differently than HUD's current data sources, including the decennial census. You asked us to review (1) how HUD estimates FMRs, (2) how accurate FMRs have been, (3) how ACS data may affect accuracy, and (4) other changes HUD can make to improve the estimates.
According to HUD, the typical process for estimating FMRs includes benchmarking, or developing baseline rents for each FMR area (generally county-based) using census data or other surveys for the years between censuses; adjusting those rents to bring them up to date; and seeking public comment before finalizing the numbers. HUD generally uses Consumer Price Index and telephone survey data to adjust baseline rents--that is, to account for rent changes since data used for baseline estimates were collected and to project the estimates into the next fiscal year (when they will be in use for subsidy purposes). HUD then lists the proposed FMRs in the Federal Register for public comment. These comments can lead to changes in FMRs, but only when they include new data or lead HUD to conduct a new survey. About 69 percent of all areas had FMR estimates in use in 2000 that were within 10 percent of rents indicated by the 2000 decennial census--the most accurate comparison data available for each FMR area. This represents an improvement over HUD's 1990 estimates. Similarly, about 73 percent of 153 areas whose FMRs HUD rebenchmarked after 2000 were within 10 percent of rents derived from recent surveys. In general, GAO found that areas that are rebenchmarked with more recent data tended to have FMRs in the most accurate range (within 10 percent). Using ACS data could improve the accuracy of FMRs by allowing HUD to benchmark more areas more frequently than is possible with current data sources, using more recent data--a factor that GAO's analysis suggests is related to accuracy. HUD's first use of ACS data will be to update existing baseline estimates for the fiscal year 2006 FMRs; HUD expects to use ACS data to set baseline rents for some fiscal year 2008 FMRs. HUD could improve its FMR estimation process by consistently following its guidelines relating to the transparency of FMRs and ensuring that it can assess the accuracy of ACS-based FMRs. Transparency would be improved by fully documenting the estimation process so that FMRs can be independently reproduced. Even ACS-based FMRs may not always be accurate, and HUD's policies require mechanisms to correct information it disseminates.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-05-342, Rental Housing: HUD Can Improve Its Process for Estimating Fair Market Rents
This is the accessible text file for GAO report number GAO-05-342
entitled 'Rental Housing: HUD Can Improve Its Process for Estimating
Fair Market Rents' which was released on May 2, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Ranking Minority Member, Subcommittee on Housing and
Transportation, Committee on Banking, Housing, and Urban Affairs, U.S.
Senate:
March 2005:
Rental Housing:
HUD Can Improve Its Process for Estimating Fair Market Rents:
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-342]:
GAO Highlights:
Highlights of GAO-05-342, a report to the Ranking Minority Member,
Subcommittee on Housing and Transportation, Committee on Banking,
Housing, and Urban Affairs, U.S. Senate:
Why GAO Did This Study:
The Department of Housing and Urban Development (HUD) annually
estimates fair market rents (FMR) for standard quality rental units
throughout the United States. Among other uses, FMRs help determine
subsidies for almost 2 million low-income families in the nation‘s
largest rental assistance program. However, concerns exist that FMRs
can be inaccurate”often, too low, preventing program participants from
finding affordable housing. Also, HUD will soon derive FMRs from a new
source, the American Community Survey (ACS), which processes data
somewhat differently than HUD‘s current data sources, including the
decennial census. You asked us to review (1) how HUD estimates FMRs,
(2) how accurate FMRs have been, (3) how ACS data may affect accuracy,
and (4) other changes HUD can make to improve the estimates.
What GAO Found:
According to HUD, the typical process for estimating FMRs includes
benchmarking, or developing baseline rents for each FMR area (generally
county-based) using census data or other surveys for the years between
censuses; adjusting those rents to bring them up to date; and seeking
public comment before finalizing the numbers. HUD generally uses
Consumer Price Index and telephone survey data to adjust baseline
rents”that is, to account for rent changes since data used for baseline
estimates were collected and to project the estimates into the next
fiscal year (when they will be in use for subsidy purposes). HUD then
lists the proposed FMRs in the Federal Register for public comment.
These comments can lead to changes in FMRs, but only when they include
new data or lead HUD to conduct a new survey.
About 69 percent of all areas had FMR estimates in use in 2000 that
were within 10 percent of rents indicated by the 2000 decennial
census”the most accurate comparison data available for each FMR area.
This represents an improvement over HUD‘s 1990 estimates, as the table
below shows. Similarly, about 73 percent of 153 areas whose FMRs HUD
rebenchmarked after 2000 were within 10 percent of rents derived from
recent surveys. In general, GAO found that areas that are rebenchmarked
with more recent data tended to have FMRs in the most accurate range
(within 10 percent).
Using ACS data could improve the accuracy of FMRs by allowing HUD to
benchmark more areas more frequently than is possible with current data
sources, using more recent data”a factor that GAO‘s analysis suggests
is related to accuracy. HUD‘s first use of ACS data will be to update
existing baseline estimates for the fiscal year 2006 FMRs; HUD expects
to use ACS data to set baseline rents for some fiscal year 2008 FMRs.
HUD could improve its FMR estimation process by consistently following
its guidelines relating to the transparency of FMRs and ensuring that
it can assess the accuracy of ACS-based FMRs. Transparency would be
improved by fully documenting the estimation process so that FMRs can
be independently reproduced. Even ACS-based FMRs may not always be
accurate, and HUD‘s policies require mechanisms to correct information
it disseminates.
Accuracy of HUD‘s Fiscal Years 2000 and 1990 FMR Estimates:
[See PDF for image]
Sources: GAO analysis of HUD data (2000 figures) and HUD (1990
figures).
[End of table]
What GAO Recommends:
To improve the usefulness of its FMR estimates, GAO recommends that HUD
fully document its methods for estimating FMRs by following all of its
data quality guidelines; use, to the extent possible, state-level ACS
data to update the fiscal year 2006 FMRs; and develop a mechanism to
assess the accuracy of future FMRs. In response to a draft of this
report, HUD agreed to better document methods for estimating FMRs and
said it is exploring options to assess accuracy.
www.gao.gov/cgi-bin/getrpt?GAO-05-342.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact David G. Wood at (202)
512-8678 or woodd@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
HUD Estimates FMRs by Defining Housing Markets, Choosing Data Sources,
Updating Rent Data, and Evaluating Public Input:
Most FMR Estimates Were Accurate within 10 Percent of the Census or
Other Rebenchmarking Surveys:
ACS Could Improve the Accuracy of FMRs by Providing HUD with More
Recent, Better Data:
HUD Did Not Follow One of Its Data Quality Guidelines and May Lack Data
Sources to Assess the Accuracy of Future FMRs:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Comments from the Department of Housing and Urban
Development:
Appendix III: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Staff Acknowledgments:
Tables:
Table 1: Accuracy of HUD's Fiscal Years 2000 and 1990 FMR Estimates
Compared with Rents from Census:
Table 2: Accuracy of HUD's FMR Estimates Compared with Rents from RDD
Surveys (by Reason for Survey, 2001-05):
Table 3: Accuracy of FMR Estimates in 2000 Compared with Rents from
Census (Based on Age of Baseline FMR Data):
Table 4: Accuracy of FMR Estimates in 2000 Compared with Rents from
Census (Based on Type of Rebenchmarking Survey):
Table 5: Accuracy of FMR Estimates in 2000 Compared with Rents from
Census (by Type of Update Factor):
Figures:
Figure 1: Example of 40th Percentile of Rent:
Figure 2: HUD's Typical Process for Estimating FMRs12:
Figure 3: HUD Regions:
Figure 4: Accuracy of HUD's Fiscal Year 2000 FMR Estimates:
Figure 5: Scope of ACS Rebenchmarking as Related to FMR Area Size and
Housing Choice Voucher Program Data:
Abbreviations:
ACS: American Community Survey:
AHS: American Housing Survey:
BAH: basic allowance for housing:
CPI: Consumer Price Index:
DOD: Department of Defense:
FMR: fair market rent:
HOPWA: Housing Opportunities for Persons with AIDS:
HUD: Department of Housing and Urban Development:
LIHTC: Low Income Housing Tax Credit:
NAS: National Academy of Sciences:
OMB: Office of Management and Budget:
PHA: public housing agency:
RDD: random digit dialing:
SRO: Moderate Rehabilitation Single-Room Occupancy:
Letter March 31, 2005:
The Honorable Jack Reed:
Ranking Minority Member:
Subcommittee on Housing and Transportation:
Committee on Banking, Housing, and Urban Affairs:
United States Senate:
Dear Senator Reed:
The Department of Housing and Urban Development's (HUD) Housing Choice
Voucher Program, commonly known as "Section 8" tenant-based assistance,
is the largest ongoing rental assistance program in the United States,
serving almost 2 million families with a budget of about $16.9 billion
for fiscal year 2005. The Housing Choice Voucher Program provides
subsidies to help low-income families afford rental housing in the
private market. To determine the amounts of the subsidies it will
provide to low-income families under the Voucher Program, and for other
purposes, HUD annually estimates fair market rents (FMR)--that is, rent
estimates that include utilities. From time to time, public housing
agencies and others have expressed concern that FMR estimates can be
inaccurate--often, too low--thereby preventing voucher holders from
being able to find affordable housing in certain areas.
HUD estimates FMRs for all bedroom size units for each area in the
entire United States (typically, counties) in advance of the year
during which they will be effective. HUD currently uses rent data from
a variety of surveys--the Bureau of the Census' decennial census long
form is the major survey used--as a baseline (or benchmark) for
estimating FMRs throughout the country.[Footnote 1] Between censuses,
HUD's practice has been to rebenchmark census-based FMRs with data from
the American Housing Survey (AHS), a Census Bureau survey performed in
certain metropolitan areas every few years, and from Random Digit
Dialing (RDD) surveys, telephone interviews that gather rent and other
data for estimating FMRs for a limited number of metropolitan and
nonmetropolitan areas annually, conducted by HUD contractors. However,
a new Census Bureau product, the American Community Survey (ACS), is
replacing the decennial census long form and will become the major
source of rent data for FMR estimates in every area. With the ACS, the
Census Bureau will publish results annually based on 1-, 3-, or 5-year
averages, depending on the population size of the area surveyed, rather
than every 10 years. For example, HUD will receive 1-year average data
(the average of 12 months) annually for areas in which the majority of
voucher holders reside.
You asked us to review HUD's process for estimating FMRs and the impact
that the incorporation of the ACS could have on the accuracy of FMRs.
Our report discusses (1) how HUD estimates FMRs, (2) how accurate HUD's
FMR estimates have been, (3) how and when the use of ACS data to
estimate FMRs may affect their accuracy, and (4) the potential for
other changes HUD could make to improve the way it estimates FMRs and
their accuracy.
To determine the general process for how HUD estimates FMRs, we
analyzed statutes, regulations, and agency documents and interviewed
HUD officials. To determine how accurate FMR estimates were, we
compared all two-bedroom FMRs that HUD put in effect for fiscal year
2000 with census data for the same year because (1) the decennial
census rent estimates are considered to be the closest estimates of the
true value of those rents and (2) HUD estimates FMRs for other bedroom
sizes as a multiple of the FMR it sets for two-bedroom units. We also
compared HUD's estimated FMRs in effect during fiscal years 2001-05 for
selected geographic areas with rents estimated using data from surveys
HUD and others conducted over this period. After making these
comparisons, we performed an associative analysis--that is, we analyzed
specific components of (or data inputs to) the FMR estimation process
to see how they might relate to the accuracy of FMRs. To determine how
and when HUD will use ACS data to estimate FMRs and what their
potential effects on the accuracy of FMRs would be, we compared ACS
with the other major surveys HUD uses to estimate FMRs, identified
salient characteristics of the ACS data, and reviewed HUD's plans for
using ACS data. To determine other changes HUD could make to improve
its estimation process and the accuracy of FMRs, we analyzed data
quality guidelines and then assessed HUD's estimation process against
the guidelines. We also interviewed officials from HUD headquarters and
field offices, as well as experts and researchers who routinely work
with housing data sources. Appendix I provides additional details on
our objectives, scope, and methodology.
Throughout this report, we refer to the "quality" of surveys or the
"quality" of data. We use quality as an overarching term for important
characteristics related to the accuracy, recency, and relevance of data
sources and surveys. Specifically, for purposes of this report, we
describe quality data obtained from surveys as:
* "accurate" when all types of rental housing units have a chance of
being selected for the survey and the sample size is large enough to
provide a 90 or 95 percent likelihood that the survey's estimates will
be within 5 to 10 percent of what would be found if the entire
population (i.e., all rents) were known;
* "recent" to the extent that the time between when data are collected
and subsequently used is minimized; and:
* "relevant" when surveys collect, at a minimum, data on rents for
HUD's program purposes and, among the survey data sources available,
HUD chooses the survey that most closely corresponds to the FMR area.
These characteristics generally match those in data quality guidelines
used by other federal agencies, and the characteristics of data or
survey quality required by HUD through statute, regulations, or
guidance for data submissions.
We conducted our work in Washington, D.C., between May 2004 and
February 2005 in accordance with generally accepted government auditing
standards.
Results in Brief:
According to HUD, the typical process to estimate FMRs includes
developing baseline rents from what it judges to be the best rent data
available for each area, adjusting those rents to bring them up to
date, and seeking public comment on its estimates prior to publishing
them for public housing agencies and others to use. Once HUD determines
the FMR areas, it uses decennial census housing data when they are
first released to establish baseline rent estimates, or benchmarks, for
each. For subsequent years, HUD uses data from other surveys--either
the AHS or RDD surveys--to establish a new baseline, or to
"rebenchmark" FMRs for certain areas. To compensate for the time lag
between when data are collected and when HUD first uses them, HUD
annually adjusts its baseline estimates in two ways. First, HUD updates
the estimates to December 31 of the current fiscal year using annual
percentage changes in rent and utility costs from the local Consumer
Price Index for major metropolitan areas, or similar information from
RDD surveys for other areas. Second, to make FMRs relevant for the
fiscal year in which they will be in effect, HUD trends, or projects,
the updated figure to the midpoint of the next fiscal year by applying
a national estimate of annual rent increases between the censuses from
the decennial census data. After making these adjustments, HUD
publishes the proposed FMRs in the Federal Register for public comment.
Although HUD considers all of the comments it receives, it typically
changes the proposed FMRs only if the comments are supported by data
that meet HUD's standards. After the period of 60 days to comment on
the Federal Register ends, HUD still considers other requests and
submissions throughout the year.
Over two-thirds of FMRs that HUD estimated for fiscal year 2000, as
well as those it estimated for areas rebenchmarked after 2000, were
within 10 percent of the rents indicated by a subsequent quality
survey, such the AHS. For example, when we compared the fiscal year
2000 FMRs (which HUD estimated in 1999) with rents from the 2000 census
data that were collected during the same period the FMRs were in
effect, 69 percent of all of HUD's FMR area estimates were within 10
percent of the census figure--an improvement over HUD's 1990 estimates,
when 39 percent of areas were within 10 percent of the 1990 census.
When we compared a limited number of FMRs that HUD estimated after 2000
with rents indicated by data from the AHS or RDD surveys that HUD or
public housing agencies (PHA) subsequently conducted, a similar
proportion of FMRs (73 percent) fell in the most accurate range. While
our associative analysis did not demonstrate what factors definitively
cause accuracy or how much each contributes, it did show that when HUD
used more recent, relevant data taken from a higher quality survey than
some HUD used to rebenchmark in the past, FMR estimates were more often
within 10 percent of the rents derived from a rebenchmarking survey.
For example, FMR estimates from areas based on more recent survey data-
-within 1 to 4 years--produced a significantly higher proportion of
FMRs that were within 10 percent of rents derived from the census than
FMR estimates from areas surveyed less recently.
The ACS could improve the accuracy of FMR estimates because it is a
higher quality survey than some HUD has used in the past and provides
more recent and local data than are currently available--beginning in
fiscal year 2006 when HUD first uses ACS data to update FMRs, and
subsequently in fiscal year 2008 when it will likely rebenchmark FMRs
in certain areas. HUD will be able to use ACS data to rebenchmark FMRs
annually (or every 3 or 5 years for areas with smaller populations),
doing so in generally the same way it used the decennial census to
estimate baseline rents. Certain challenges related to the manner in
which ACS data are processed and reported may affect FMR accuracy. For
example, ACS data are averages of monthly survey data, which may
"smooth" rental market shifts or trends. According to HUD officials,
they will begin to address these challenges when the Census Bureau
releases the fiscal year 2005 data (in Fall 2006), the data collected
during the first year of full implementation for the ACS. Despite the
challenges in using the data, neither we nor experts and researchers
who routinely work with housing data sources identified viable
alternatives to the ACS.
Potential exists for HUD to improve its estimation process for FMRs and
their accuracy because the agency (1) presently does not follow its
objectivity guideline for ensuring the transparency and reproducibility
of its FMR estimates and (2) may in the future lack a way to assess the
accuracy of ACS-based rent estimates. HUD, like other federal agencies,
has developed guidelines to ensure that it disseminates quality data.
HUD's guidelines include ensuring the utility (usefulness), integrity
(protection from unauthorized access), and objectivity (transparency
and reproducibility) of data. Of the three, HUD appears to be following
the utility and integrity guidelines as they relate to the FMR
estimation process. For example, HUD meets its utility guidelines by
estimating FMRs on an annual schedule and making the estimates public
and easily accessible. HUD does not follow one of these three--its
objectivity guideline--because it has made neither the data it uses nor
its methods for estimating FMRs sufficiently transparent for an
independent party, such as GAO, to be able to substantially reproduce
FMRs using publicly available information. Finally, as HUD transitions
to ACS-based FMRs, it will not only stop using the decennial census
long form but it will rely less on RDD surveys and the AHS because of
cost and quality concerns about these surveys. As a result, HUD may not
have a means to assess the accuracy of future FMR estimates once it
relies almost exclusively on the ACS.
This report contains recommendations designed to improve HUD's
processes for estimating FMRs and their accuracy. We provided HUD with
a draft of this report for its review and comment. HUD agreed that it
can better document its methods for estimating FMRs and described
efforts it has under way to improve the transparency and
reproducibility of its methods. HUD also requested that we clarify
certain transparency and reproducibility issues in our report and
recognize its ongoing efforts. HUD disagreed with our recommendation to
use state-level ACS data in fiscal year 2006 FMRs, stating that it has
concerns about the adequacy of ACS sample sizes for the fiscal year
2006 estimates. We have retained this recommendation because it
contains a caution that HUD should do so as much as possible, but only
in instances where HUD determines that the ACS data are sufficiently
reliable for this purpose. HUD did not explicitly state whether it
agrees or disagrees with our recommendation that it develop a mechanism
to assess the accuracy of future FMRs, but it did indicate that it
recognizes there are areas, such as those with unusual rent increases
or decreases, that could experience FMR estimation errors when HUD uses
ACS data for its estimates. HUD also indicated that it anticipates
continuing to review AHS surveys and making limited use of RDD surveys
while it explores other long-term alternatives for assessing the
accuracy of FMRs. Because HUD recognized the challenge we pointed out
relating to the accuracy of FMRs and stated that it is currently
exploring ways to address this issue, we have retained our
recommendation. HUD also suggested a number of technical clarifications
to our report, which we have made, as appropriate.
Background:
HUD estimates FMRs in order to set upper and lower bounds on the cost
and quality of typical, standard quality units voucher holders rent
and, in doing so, ensure that the units rented are modest (not
luxurious), meet the housing quality standards HUD sets for them, and
are available in sufficient numbers to those seeking to use the
vouchers. Local PHAs use FMRs to set payment standards, which are the
basis for determining the subsidies HUD provides to help low-income
families afford housing in the private rental market under the Housing
Choice Voucher Program. Specifically, PHAs may set payment standards at
90 to 110 percent of the FMR for their area and, with HUD approval,
above 110 percent of the FMR. Because HUD generally requires voucher
holders to contribute 30 percent of their income as rent, the amount of
HUD's subsidy (the rental assistance) then becomes the difference
between the PHA's payment standard and 30 percent of the family's
monthly income.[Footnote 2]
While FMRs are primarily used in the Housing Choice Voucher Program,
other programs both inside and outside of HUD also use FMRs. For
example, HUD uses FMRs to:
* determine initial rents for housing assistance payments in the
Moderate Rehabilitation Single-Room Occupancy program;[Footnote 3]
* determine initial renewal rents for units in some expiring project-
based "Section 8" contracts under the Mark-to-Market Program;[Footnote
4]
* set maximum rents under the HOME Program;[Footnote 5]
* set standard rent ceilings in the Housing Opportunities for Persons
with AIDS (HOPWA) Program;[Footnote 6]
* make calculations for the "difficult development" areas under the Low
Income Housing Tax Credit (LIHTC) Program;[Footnote 7] and:
* review the feasibility of proposed LIHTC projects.
The Department of Defense (DOD) compares its basic allowance for
housing (BAH) amounts, which is housing assistance it provides military
personnel, to HUD's FMRs. More specifically, when DOD determines that
it is not cost-effective to collect proprietary survey data on housing
costs, it uses FMRs as a basis for calculating comparable figures.
Whatever its programmatic use, an FMR must fall within certain
statutory and regulatory parameters. The U.S. Housing Act of 1937, as
amended, requires HUD to base FMRs on the most recent available data to
estimate rents of various sizes and types within a market.[Footnote
8]HUD regulations and guidance on FMRs further emphasize that rent
survey data must be the most accurate and current available.[Footnote
9] HUD specifically requires that the survey methodology provide
statistically reliable, unbiased estimates of gross rents by, among
other things, having a large enough sample so that there is a 95
percent likelihood that the survey's estimates will be within 5 to 10
percent of what would be found if the entire population (i.e., all
rents) were collected. HUD also requires that survey samples be random
and reflect rent levels that exist for housing units of different ages,
types, and geographic locations within the entire FMR area. Using these
considerations, HUD's three primary data sources for FMRs are the
decennial census (long form), the AHS, and RDD surveys. A RDD survey is
a computer-aided telephone survey of randomly selected households that
may be conducted by HUD, individual PHAs, or others.
Finally, FMRs are specifically defined as annual estimates of the 40th
percentile of gross rents for typical, nonsubstandard market-rate
rental units occupied by recent movers.[Footnote 10]
Fortieth Percentile of Rents:
The 40th percentile is the point in a distribution of numbers at which
40 percent of the numbers are at or below that point; for FMR purposes,
this is the dollar amount below which 40 percent of the standard
quality rental units in an area have rented. For example, in the
distribution in figure 1, $670 is the 40th percentile because 4 of the
10 rents are at or below that point:
Figure 1: Example of 40th Percentile of Rent:
[See PDF for image]
[End of figure]
Gross Rent:
A gross rent is the rent a tenant pays to the owner--sometimes called
"shelter" costs--plus the cost of utilities (usually, electricity, gas,
water and sewer, and trash removal charges, but not telephone service).
If utilities are included in the rent, then the gross rent is simply
the amount paid to the owner.
Typical, Standard Rental Units:
By statute, FMRs are estimates of market rents for typical, standard
quality housing. HUD has determined that certain rental units should be
excluded from its data sources in order to meet this definition.
Specifically, these include rents for units built within the last 2
years (which tend to be higher priced); units receiving some form of
subsidy (such as public housing) where the rent does not reflect a
"market" price; and substandard units--for example, units without
adequate heating or plumbing--that likely would not meet the housing
quality standards applicable to the voucher program.[Footnote 11]
Recent Movers:
HUD has found that rents for units occupied by recent movers (i.e.,
tenants who moved within the past 15 to 24 months) are typically higher
than what other renters pay. By linking FMR estimates to the rents that
recent movers have paid, HUD tries to ensure that they more closely
reflect the rents that low-income households new to the voucher program
may face when they look for rental housing.
The Census Bureau is discontinuing the long form and has begun
replacing it with the ACS.[Footnote 12] Overall, the ACS will provide
the same type of data as the decennial census long form at the same
level of geographic area detail, but in a more timely way because it
will be an ongoing survey (as opposed to one conducted every 10 years).
Specifically, the ACS will collect data monthly and each year publish
either 1-, 3-, or 5-year averages (depending on the population in each
area).[Footnote 13]
HUD Estimates FMRs by Defining Housing Markets, Choosing Data Sources,
Updating Rent Data, and Evaluating Public Input:
According to HUD, the typical process it uses to estimate FMRs (rent
estimates that include utilities) includes choosing what it judges to
be the best rent data available for each area, adjusting those data so
that they are up to date, and seeking public comment on the estimates
prior to finalizing them for public housing agencies and others to use
(see fig. 2). Once HUD determines each FMR area and receives decennial
census data or AHS or RDD data, it analyzes the rent data to establish
a "benchmark" FMR for each area by determining the 40th percentile of
the rent distribution. Then, HUD annually adjusts the estimates to
reflect changes in rent and utility costs to compensate for the lag
between data collection and the period in which the FMR will be in
effect. After adjusting the FMR for each area, HUD publishes the
proposed FMRs for public comment. Although HUD considers all of the
comments it receives, it typically changes FMRs only if the comments
are supported with data that meet HUD's standards. The public can also
affect FMRs by (1) requesting that HUD conduct an RDD for the area or
(2) submitting comments with supporting rent data or information that
causes HUD to conduct additional research.
Figure 2: HUD's Typical Process for Estimating FMRs:
[See PDF for image]
[End of figure]
HUD Establishes Areas, Uses Survey Data to Benchmark and Adjust FMRs:
To ensure that the FMR estimates are useful, HUD's first step is to
determine FMR areas that they believe correlate with distinct housing
markets, typically the size of a county (see fig. 2). To determine FMR
areas, HUD generally uses the boundaries of Office of Management and
Budget (OMB)-defined metropolitan and nonmetropolitan areas.[Footnote
14] According to HUD, it may also create new areas that do not
correspond to OMB boundaries, particularly within sprawling
metropolitan areas that may have separate housing markets. For
instance, HUD created a separate FMR area for West Virginia counties
that had been included in OMB's Washington, D.C., metropolitan area,
because HUD did not consider these counties to be part of the
Washington housing market. Although HUD may revise FMR area definitions
at any time, it typically does so infrequently (not every year when it
develops FMRs).[Footnote 15] HUD publishes FMR estimates annually for
356 metropolitan FMR areas and 2,303 nonmetropolitan FMR areas in the
United States, Puerto Rico, the Virgin Islands, and Guam.
HUD's second step is to benchmark--that is, estimate baseline rents--
for two-bedroom units by identifying the 40th percentile of the
estimated rent distribution for each area with the most recent
available data (for FMR areas for which no new, recent rent data are
available, HUD skips this step and updates the existing FMR). HUD
chooses from a variety of data for benchmarking, including the
decennial census, the AHS, RDD surveys, and traditional surveys from
the public. According to HUD officials:
* The decennial census provides the highest quality data to estimate
FMRs because it provides (1) rent estimates within 1 percent of the
true value of the 40th percentile of rents in metropolitan areas and
(2) the most consistent data for all areas to establish a baseline for
FMRs once every 10 years.
* Data from RDD surveys have sufficient quality to meet HUD's
requirements and provide estimates within 3.5 to 5 percent of the true
value of rents for a limited number of areas, usually metropolitan
areas.
* The AHS offers sufficient quality data with estimates within 7
percent of the true value of rents the survey is measuring and are
available for a limited number of metropolitan areas every few years.
According to HUD officials, to be consistent with the definition of
FMRs, HUD only uses survey data for rental housing units that are:
* nonsubsidized and of "standard" quality;[Footnote 16]
* more than 2 years old;
* nonseasonal (i.e., occupied year round);
* located on properties of less than 10 acres; and:
* leased by recent movers (those who have moved within the last 15 to
24 months).
HUD adds estimated utility costs to the base rent estimates it derives
from RDDs because these surveys do not include that information. To do
so, HUD officials estimate the cost of utilities with PHA utility
schedules, which include a list of average monthly costs for each
utility. The decennial census and AHS data include utilities in their
base year rent estimates.
The third and fourth steps in the process involve adjusting FMRs. To
mitigate the time lag between data collection and FMR use, HUD first
updates FMRs to December 31 of the current fiscal year with information
about changes in the rent and utility index from the Consumer Price
Index (CPI) program for specific metropolitan areas or, for other
metropolitan and all nonmetropolitan areas, with the gross rent "change
factors" established by regional RDD surveys. To estimate the gross
rent change factor, or the measure of rent change, HUD conducts
regionwide RDD surveys in each of its 10 multistate regions (see fig.
3).
Figure 3: HUD Regions:
[See PDF for image]
[End of figure]
Once FMRs are updated, HUD attempts to make them useful for the fiscal
year in which they will be in effect by trending, or projecting, them
to the midpoint of that fiscal year. To do this, HUD uses a national
measure of annual rent increases (i.e., average rent increases during
the 10 years between the censuses, typically 3 percent, on the basis of
decennial census rent data).
In the fifth step, HUD also estimates FMRs for other bedroom sizes (in
practice, one-, three-, and four-bedroom). Because HUD usually lacks
sufficient survey data to directly estimate FMRs for all unit sizes, it
typically benchmarks FMRs for two-bedroom units only and estimates rent
ratios for other sizes.[Footnote 17] According to HUD officials, these
ratios are based on local rent relationships derived from decennial
census rent data. Once HUD calculates these ratios, it ensures that
they are "sequential," which means that FMRs increase as unit size
increases (e.g., in 1994, three-bedroom FMRs had to be at least 125
percent of two-bedroom FMRs, and four-bedroom FMRs had to be at least
140 percent of two-bedroom FMRs). After HUD estimates FMRs for each
bedroom size unit, it applies a "bonus" to increase FMRs for larger
units (three-bedrooms or larger) to help ensure that the units can be
rented by voucher holders.
HUD Provides Opportunities for Public Input on Proposed FMRs as Well as
Those in Effect:
To provide for public input on proposed FMRs:
* HUD publishes the proposed FMRs in the Federal Register to solicit
public comments, usually in April or May of each year (sixth process
step).
* The public submits comments during the (approximate) 60-day public
comment period.
* After the comment period, HUD reviews the responses received and may
act on some of them prior to finalizing FMRs and publishing them again
in final form in the Federal Register in September (seventh process
step). FMRs are in effect for the next fiscal year, which starts
October 1.
After the period of 60 days to comment on the Federal Register ends, to
address situations in which existing FMRs are perceived to be
inaccurate, members of the public--often, PHAs--also can submit
information on the existing FMR for HUD to consider. For example, PHAs
can at any time conduct and submit to HUD the results of their own RDD
surveys; HUD applies the same criteria to these surveys as it does to
those that PHAs submit in response to the proposed FMRs in the Federal
Register. Specifically, HUD requires that any PHA-submitted data it
uses to change FMRs must be statistically reliable; unbiased estimates
of gross rents; and, among other things, have a large enough sample
that there is a 95 percent likelihood that the survey's estimates will
be within 10 percent of what would be found if the entire population
(i.e., all rents) were collected.[Footnote 18] Also, PHAs may at any
time outside of the formal comment process request that HUD conduct an
RDD survey or submit information about the existing FMR that may cause
HUD to conduct additional research.
While the Quality Housing and Work Responsibility Act of 1998 gave PHAs
the flexibility to set payment standards at 90 to 110 percent of their
FMRs, they may also request an exception to further adjust either the
payment standard or the FMR for their area. Specifically, when PHAs
believe that payment standards at 110 percent of the FMR are
insufficient to allow voucher holders to successfully lease units, they
may request from HUD one of two possible exceptions: (1) increase the
payment standard to exceed the FMR by more than 10 percent or (2)
benchmark the FMR estimate at the 50TH percentile of rent for the area,
rather than the 40th percentile of rent.[Footnote 19]
Most FMR Estimates Were Accurate within 10 Percent of the Census or
Other Rebenchmarking Surveys:
According to our analysis, more than two-thirds of (1) all FMRs that
HUD estimated for fiscal year 2000 and (2) a limited number of FMRs
that HUD rebenchmarked after 2000 were within 10 percent of the rents
derived from subsequent surveys such as the census, the AHS, or an RDD
survey. Specifically, 69 percent of all of HUD's FMR estimates for
fiscal year 2000--published in 1999--were within 10 percent (the most
accurate range) of rent estimates derived from the 2000 census.
Moreover, when we considered FMRs by type of area, FMR estimates for 86
percent of metropolitan areas and 66 percent of nonmetropolitan areas
fell in the most accurate range in 2000. Similarly, when we compared
rents derived from rebenchmarking surveys done for 153 FMR areas since
2000 with the FMR estimates in place at the time of the rebenchmarking
survey, 73 percent of the estimates were within 10 percent of the rents
derived from the surveys. FMR estimates were more often associated with
accuracy when HUD based them on data that were more recent, taken from
a higher quality survey than some HUD has used in the past, or more
relevant because the survey covered an area closely matching the
boundaries of the FMR area.[Footnote 20] Other factors not related to
the specific survey HUD used to estimate FMRs, such as difficulty in
estimating utility costs, may also affect the accuracy of FMR
estimates.
Over Two-thirds of All FMRs for 2000 Were Accurate within 10 Percent of
Rents Derived from the 2000 Census:
According to our analysis, for fiscal year 2000, 69 percent of FMRs
that HUD estimated for fiscal year 2000 were within 10 percent of the
2000 census rent estimates, the most accurate comparison data available
for each FMR area (see fig. 4).
Figure 4: Accuracy of HUD's Fiscal Year 2000 FMR Estimates:
[See PDF for image]
[End of figure]
FMR estimates that were within 10 percent of the rents derived from a
higher quality survey, such as a census or RDD survey, could be higher
or lower (i.e., within plus or minus 10 percent). For example, if HUD
estimated an FMR of $500 for an area and a higher quality survey of the
same area found a 40th percentile rent of $550, the difference is
within 10 percent of the survey as follows:
$550 (new survey estimate)-$500 (existing FMR) = $50 difference:
$50 difference/$550 (new survey estimate) = 9 percent:
In this example, the original FMR was lower than the rent indicated by
the recent survey, but was within 10 percent.
The results for 2000 are a significant improvement over results from
1990 when HUD reported that 39 percent of FMRs were in the most
accurate range (see table 1). Furthermore, arraying the data by
population to account for areas where estimates affected more potential
voucher holders shows that a greater share of FMR estimates were within
the most accurate range in 2000 than what HUD reported for 1990.
Considering FMR estimates by type of area, we also found that more
metropolitan and nonmetropolitan areas were within 10 percent accuracy
in 2000 than HUD reported in 1990.[Footnote 21]
Table 1: Accuracy of HUD's Fiscal Years 2000 and 1990 FMR Estimates
Compared with Rents from Census:
Fiscal year: 2000;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 2%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 8%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 69%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 19%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 2%.
Fiscal year: 1990;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 25%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 30%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 39%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 4%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 2%.
Weighted by population:
Fiscal year: 2000;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 2%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 4%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 88%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 6%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 1%.
Fiscal year: 1990;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 5%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 10%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 73%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 12%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 1%.
Metropolitan areas:
Fiscal year: 2000;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 3%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 6%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 86%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 4%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 1%.
Fiscal year: 1990;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 8%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 14%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 71%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 8%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 0%.
Nonmetropolitan areas:
Fiscal year: 2000;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 2%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 8%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 66%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 21%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 3%.
Fiscal year: 1990;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 29%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 31%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 34%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 4%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 2%.
Sources: GAO analysis of HUD data (2000 figures) and HUD (1990
figures).
[End of table]
As table 1 shows, our analysis indicates that in 2000, where FMR
estimates were higher or lower than the census by 10 percent or more,
most often the FMR was too low, a different result from 1990 when HUD
reported that most FMR estimates outside of the most accurate range
were too high.
Since the 2000 Census, HUD and others surveyed a limited number of FMR
areas (153, as of September 2004). When we compared the rents derived
from these surveys with FMR estimates in effect for these years, the
outcome was similar to the results we found in our comparison with the
2000 census--almost three-fourths (73 percent) of FMR estimates were
within 10 percent of the survey rents. When analyzing the 153 areas, we
also found a difference in the results shown by rebenchmarking surveys
undertaken for different reasons. HUD and PHAs conducted rebenchmarking
surveys for two basic reasons: (1) HUD was adhering to a schedule in
which it surveyed selected large metropolitan areas on a rotational
basis or (2) HUD, PHAs, or others received information suggesting FMRs
were inaccurate (usually a complaint that an FMR was too low) in a
specific area. As shown in table 2, complaint-driven surveys (RDD
surveys that were conducted by HUD following a PHA request or by a PHA
itself) more often found inaccurate FMRs (i.e., FMR estimates that were
10 percent or more different from the rents derived from the survey).
Table 2: Accuracy of HUD's FMR Estimates Compared with Rents from RDD
Surveys (by Reason for Survey, 2001-05):
Reason for survey: HUD schedule;
Compared with RDD survey rents--percentage of FMRs that were: Higher by
20% or more: 0%;
Compared with RDD survey rents--percentage of FMRs that were: Higher
by 10% to 19.9%: 3%;
Compared with RDD survey rents--percentage of FMRs that were: Within
10%: 87%;
Compared with RDD survey rents--percentage of FMRs that were: Lower by
10% to 19.9%: 9%;
Compared with RDD survey rents--percentage of FMRs that were: Lower by
20% or more: 1%.
Reason for survey: By request (HUD surveyed);
Compared with RDD survey rents--percentage of FMRs that were: Higher by
20% or more: 2%;
Compared with RDD survey rents--percentage of FMRs that were: Higher by
10% to 19.9%: 2%;
Compared with RDD survey rents--percentage of FMRs that were: Within
10%: 66%;
Compared with RDD survey rents--percentage of FMRs that were: Lower by
10% to 19.9%: 26%;
Compared with RDD survey rents--percentage of FMRs that were: Lower by
20% or more: 5%.
Reason for survey: PHA surveyed;
Compared with RDD survey rents--percentage of FMRs that were: Higher by
20% or more: 0%;
Compared with RDD survey rents--percentage of FMRs that were: Higher
by 10% to 19.9%: 0%;
Compared with RDD survey rents--percentage of FMRs that were: Within
10%: 53%;
Compared with RDD survey rents--percentage of FMRs that were: Lower by
10% to 19.9%: 42%;
Compared with RDD survey rents--percentage of FMRs that were: Lower by
20% or more: 5%.
Source: GAO analysis of HUD data.
Note: HUD estimated FMRs we used in this comparison prior to the public
comment step that takes place after its estimation process. When HUD
received the results of RDD surveys prior to the public comment step,
it used (and published) those rent estimates rather than the initial
FMR estimate it had developed. As a result, some of the estimates we
use in this comparison were never published by HUD as proposed FMRs.
[End of table]
According to HUD, those areas it surveyed because they were on its
schedule were, like the complaint-driven RDD surveys, not random
selections. Most often, HUD selected areas from its schedule because it
had not surveyed them recently, which means that HUD tended to choose
areas for which the length of time since the last rebenchmarking survey
was longer. According to HUD officials, choosing areas for RDD surveys
for this reason increases the likelihood that these surveys would find
inaccurate FMRs. Further, to the extent that complaints are more likely
to arise when FMRs are believed to be too low, rather than too high, it
is not surprising that complaint-driven surveys were much more likely
to show rents higher than FMRs, rather than lower.
Quality Survey Data Tended to Produce FMR Estimates That Were Accurate
within 10 Percent:
Survey data that had one or all of the characteristics we summarize as
"quality"--recent, accurate, or relevant--tended to more often produce
FMRs within 10 percent of another rebenchmarking survey. Specifically,
our analysis showed that FMR estimates more often fell in the most
accurate range when HUD based FMRs on survey data that were (1) more
recent, (2) taken from a higher quality survey than some surveys HUD
has used in the past, or (3) more relevant because their source closely
matches the boundaries of the FMR area.
More Recent Data:
FMR estimates that HUD rebenchmarked with newer survey data (1 to 4
years old) were associated with greater accuracy in 2000 (see table 3).
For example, our analysis found that 88 percent of all FMR estimates
based on newer data (i.e., 1 to 4 years old) were within 10 percent of
the census estimates in 2000.
Table 3: Accuracy of FMR Estimates in 2000 Compared with Rents from
Census (Based on Age of Baseline FMR Data):
Age of baseline FMR data: 1 to 4 years;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 1%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 6%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 88%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 5%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 0%.
Age of baseline FMR data: 5 to 7 years;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 3%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 9%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 67%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 20%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 1%.
Age of baseline FMR data: No survey from 1990 to 2000;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 2%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 7%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 67%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 20%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 3%.
Source: GAO analysis of HUD data.
Note: Areas based on 2000 decennial census data or 8-, 9-, or 10-year-
old non-Census data comprised too few areas from which to calculate
separate statistics.
[End of table]
In considering the association we found between recent data and
accuracy, HUD officials stated that the length of time since the last
rebenchmarking survey likely affected the accuracy of FMR estimates. As
our analysis showed, areas for which the baseline data were older
(including those for which there was no rebenchmarking survey between
the 1990 and 2000 censuses) more often had FMR estimates that were 10
percent or more higher or lower than the estimate from a recent survey.
Data from Higher Quality Surveys:
When HUD used data from higher quality surveys than some surveys it had
used in the past, its FMR estimates were accurate more often than when
it relied on lesser-quality means, such as the traditional surveys some
PHAs conducted before HUD adopted the RDD survey methodology.
Currently, HUD uses the AHS or RDD surveys to rebenchmark FMRs between
the decennial censuses. However, until the mid-1990s, HUD also, on
occasion, accepted from PHAs and used for rebenchmarking FMRs survey
data that PHAs collected via less rigorous traditional or telephone
surveys.[Footnote 22] The AHS and RDD surveys can be considered higher
quality than the less rigorous ones HUD once accepted because they have
(1) survey characteristics required by HUD's regulations and guidelines
and (2) data from a survey closely corresponding to the boundaries of
the FMR areas. As shown in table 4, the higher quality survey sources-
-AHS and RDD surveys--more often led to FMRs within 10 percent accuracy
than the estimates based on less rigorous methods.
Table 4: Accuracy of FMR Estimates in 2000 Compared with Rents from
Census (Based on Type of Rebenchmarking Survey):
Type of last FMR rebenchmarking survey: AHS;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 5%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 0%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 95%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 0%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 0%.
Type of last FMR rebenchmarking survey: RDD-HUD;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 2%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 7%;
Compared with decennial census rents-
-percentage of FMRs that were: Within 10%: 86%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 5%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 0%.
Type of last FMR rebenchmarking survey: RDD-PHA;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 3%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 22%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 72%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 4%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 0%.
Type of last FMR rebenchmarking survey: Traditional;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 2%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 6%;
Compared with decennial census rents-
-percentage of FMRs that were: Within 10%: 65%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 26%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 1%.
Type of last FMR rebenchmarking survey: Telephone;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 11%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 28%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 61%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 0%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 0%.
Type of last FMR rebenchmarking survey: No Survey from 1990 to 2000;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 2%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 7%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 67%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 20%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 3%.
Source: GAO analysis of HUD data.
[End of table]
More Relevant (Local) Data:
When HUD used more relevant (local) surveys to update FMRs--that is, to
adjust for inflation rather than to rebenchmark or revise the baseline-
-the results were similar: FMR estimates were associated with greater
accuracy. As shown in table 5, when HUD updated FMR estimates with the
more local metro-specific CPI--a survey that generally matches the
boundaries of metropolitan FMR areas--91 percent of estimates were
within 10 percent accuracy. When HUD used regional RDD surveys--which
cover much broader areas than the FMR area boundaries--to update FMR
estimates, many fewer were within 10 percent accuracy.
Table 5: Accuracy of FMR Estimates in 2000 Compared with Rents from
Census (by Type of Update Factor):
Type of update factor: Metro-specific CPI;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 5%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 1%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 91%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 3%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 0%.
Type of update factor: RDD regional gross rent change factor;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 20% or more: 2%;
Compared with decennial census rents--percentage of FMRs that were:
Higher by 10% to 19.9%: 8%;
Compared with decennial census rents--percentage of FMRs that were:
Within 10%: 68%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 10% to 19.9%: 19%;
Compared with decennial census rents--percentage of FMRs that were:
Lower by 20% or more: 3%.
Source: GAO analysis of HUD data.
[End of table]
According to HUD officials, the use of broad factors--that is, factors
from surveys covering a larger geographic area than the FMR area--for
updating and trending in the FMR estimation process contributes to
inaccuracy in the estimates. For instance, the update factors derived
from regional RDD surveys may not capture changes in the local economy
within a specific FMR area, such as a large employer leaving town or a
sizable increase in the housing supply that may affect rents.
Furthermore, HUD officials stated that the use of a nationwide factor
for trending FMR estimates--the process of projecting FMR estimates
into the future year for which they will be effective--may not capture
local trends. (As previously noted, HUD currently applies to all FMR
areas a standard trending factor derived from the change in the
national average rents between the 1990 and 2000 censuses.)
HUD Believes That Other Factors May Influence the Accuracy of FMR
Estimates:
In addition to the factors we identified as being associated with the
accuracy of FMR estimates, HUD officials indicated several more factors
that might also affect accuracy. Specifically, these officials cited
(1) general survey error common to all such estimates, (2) the
characteristics of nonmetropolitan areas, (3) difficulty in estimating
utility costs, and (4) recent mover rent changes differing from rent
changes captured by the CPI.
General Survey Error:
The data from the survey sources HUD uses are estimates which, by
definition, can introduce error into FMR estimates. All surveys are
subject to various types of error, which means that survey data may not
precisely match the true value the survey is trying to measure. For
example, sampling error occurs because a sample rather than an entire
population was surveyed, and, according to HUD officials, census data
for FMR estimates are generally subject to a 1 percent sampling error
(in metropolitan areas). While HUD considers census data to be the best
source for rent estimates (primarily because these data have a far
larger sample size than any other source used), even the census
includes some areas with low sample sizes or low response rates.
Characteristics of Nonmetropolitan Areas:
Our analysis showed that FMR estimates for nonmetropolitan areas were
less likely to be based on quality data (more recent, taken from a
higher quality survey and more relevant) and were also less likely to
be more accurate. HUD officials told us that nonmetropolitan areas are
a lower priority for rebenchmarking surveys between the censuses
because they believe it is better to focus their limited resources (for
their own rebenchmarking RDD surveys) on the areas where more potential
voucher holders live (i.e., the metropolitan areas). Nonmetropolitan
areas were less likely to have a recent rebenchmarking survey
(sponsored by HUD)--between 1990 and 2000, HUD rebenchmarked 73 percent
of all metropolitan areas and 31 percent of all nonmetropolitan areas.
Also, HUD updates almost all nonmetropolitan areas using the broad
update factors it derives from its regional RDD surveys, meaning that
these areas' FMR estimates are updated with data that are less "local"
than what HUD applies to the larger metropolitan areas with local CPI
rent change estimates. Additionally, surveys of nonmetropolitan areas
(even the census) often have relatively lower sample sizes than
metropolitan areas, affecting the quality of the data for
rebenchmarking FMR estimates there and, as a result, the accuracy of
these estimates.[Footnote 23]
Difficulty in Estimating Utility Costs:
According to HUD officials, utility cost data are a source of error in
all three survey data sources HUD uses to estimate FMRs. For example,
renters have been documented as unreliable sources of the utility costs
they pay, yet the census relies on them to report utility cost
estimates. Utility costs for RDD surveys come from a utility cost
schedule supplied by the local PHA; however, according to HUD
officials, although PHAs certify that the data are correct, utility
schedules can be unreliable and introduce bias into FMR
estimates.[Footnote 24] The AHS uses a utility estimation model
(consisting of certain survey variables) that HUD officials believe
corrects to some extent for the error introduced by relying on tenant
reporting. Nonetheless, they noted that the AHS model is based on
survey estimates and thus remains subject to error in ways common to
all surveys.
Recent Mover Rent Changes in Metropolitan Areas:
HUD officials told us that the local survey HUD uses for updating FMRs
in some metropolitan areas--the metro-specific CPI--may not capture
sudden changes in rents for recent movers. According to HUD, CPIs
measure overall rent changes for all renters in a fixed group of units.
However, rent changes for recent movers can be significantly different
from changes for all renters. For example, HUD officials stated that
San Francisco and Boston are among the more volatile housing markets in
the country and, as a result, among the most difficult for which to
estimate FMRs. Specifically, in 2000 and 2001, San Francisco's recent
mover rents increased sharply, then decreased suddenly in 2002.
However, the CPI for San Francisco, which covers all renters, showed
above-average but not exceptional rent increases in 2000 and 2001 and
no change for 2002.
ACS Could Improve the Accuracy of FMRs by Providing HUD with More
Recent, Better Data:
The ACS, which is replacing the decennial census long form, could
improve the accuracy of FMRs because it is a higher quality survey
(compared with others HUD has available between the decennial censuses)
and it provides more recent data that closely matches the boundaries of
HUD's FMR areas. HUD plans to begin to use ACS data for fiscal year
2006 FMRs. However, certain challenges that we and others, including
the National Academy of Sciences (NAS), have identified may affect the
extent to which HUD can use ACS data to improve its estimates. County-
level ACS data, which will be available each year, could increase the
accuracy of FMRs because HUD plans to use them to rebenchmark all areas
more frequently. Because the ACS data are more recent than the
decennial census data and generally of similar quality and content, HUD
plans to use ACS data to rebenchmark FMRs in generally the same way
that it used the decennial census data in the past, but it will be able
do so more frequently. Certain challenges for HUD regarding the ways
ACS data are processed and reported may affect its plans for using
them. For example, the Census Bureau averages ACS data over 1-, 3-, and
5-year time periods, and averaging could mask sharp trends in rents
because it can smooth changes that occur within the time period. HUD
plans to address these challenges after it receives fiscal year 2005
ACS data--the data collected during the first year of full ACS
implementation--in Fall 2006. Despite the challenges in using these
data, neither we nor experts and researchers who routinely work with
housing data sources identified viable alternatives to the ACS.
The ACS Is a Higher Quality Survey That Provides More Recent and Local
Data:
The ACS could improve the accuracy of FMRs because it is a higher
quality survey than HUD currently has available between the decennial
censuses and it provides more recent data closely matching the
boundaries of HUD's FMR areas.
Higher Quality Survey:
The ACS is of higher quality than data sources (RDD surveys and the
AHS) currently available to estimate FMRs between the decennial
censuses. According to the Census Bureau, like its long-form
predecessor, the ACS is the highest quality household survey currently
conducted by the Census Bureau, and it will provide data more
frequently.[Footnote 25] The ACS derives similar information as the
decennial census long form, and its results undergo stringent
processing by the Census Bureau. Moreover, according to HUD officials,
the ACS is an impressive improvement over data from any other source.
For instance, although the AHS is also a Census Bureau product, it is
similar to RDD surveys because it provides data for only a
comparatively small number of areas and does so less frequently. More
specifically, the AHS covers a limited number of the largest
metropolitan areas every few years.
More Recent Data:
Using ACS data to estimate FMRs could improve their accuracy because it
provides HUD with more recent county-level data. More specifically, the
ACS will provide data each year that is based on 1-, 3-, or 5-year
rolling averages (i.e., the Census Bureau will collect data monthly,
average them over 12 months, and publish new 1-, 3-, and 5-year
averages each year). Because our analysis indicated that FMRs estimated
with recent data (i.e., data that are 4 years old or less) more often
tended to be within 10 percent of the results of a rebenchmarking
survey, FMRs estimated with annual and 3-year average data could be
more accurate. Even though FMRs estimated with 5-year average data
would be based on some data older than 4 years, they could also be more
accurate than is now the case because HUD could rebenchmark them every
5 years (as opposed to the 10 years between censuses).
More Local Data:
The ACS also will provide more local data--more specifically, state-
level data--that HUD could use to update FMRs and therefore lead to
more accurate FMRs. Currently, HUD updates FMRs for the majority of
areas with regional RDD surveys, each of which provides HUD with
aggregate gross rent change estimates based on data from up to eight
states. As previously noted, our analysis suggested that when HUD
estimated FMRs with more local data (i.e., data from a survey that
closely corresponds to the boundaries of the FMR areas) more FMRs fell
within the most accurate range. As a result, annual state-level ACS
data could enable HUD to more accurately update FMRs. Although the
state-level data do not closely correspond to the boundaries of FMR
areas, they cover areas much smaller than the currently used RDD
surveys.
HUD Expects to First Use ACS Data to Update Fiscal Year 2006 FMRs:
HUD expects to first use ACS data to update its estimated baseline
rents when preparing fiscal year 2006 FMRs. To do so, HUD plans to use
regional-level ACS data, rather than the more local state-level ACS
data that will be available to it. The state-level ACS data would
provide reliable data for geographic levels smaller than the areas
covered by the regional ACS (or regional RDD surveys). However,
according to HUD officials, they believe they need to obtain and work
with the ACS data, assuring themselves of its reliability and
usefulness before they will consider updating FMRs with the state-level
ACS data.
The effect of ACS data on FMR accuracy could be most notable once HUD
begins to rebenchmark--not just update--FMRs with these data, which
will likely begin with the fiscal year 2008 FMRs. HUD will use the
first data available under ACS full implementation in Fall 2006 to
rebenchmark fiscal year 2008 FMRs and plans to use them in ways similar
to how it had used decennial census data because their content and
quality are similar to that of the decennial census data. Figure 5
describes how often HUD could rebenchmark different-sized areas with
ACS data, showing that, for example, HUD will likely rebenchmark FMRs
for large metropolitan areas--where the most potential voucher holders
live--every year.
Figure 5: Scope of ACS Rebenchmarking as Related to FMR Area Size and
Housing Choice Voucher Program Data:
[See PDF for image]
Note: The most recent available data for population and number of
housing choice vouchers per FMR area are from fiscal years 2000 and
2003, respectively. We estimated the "voucher dollar" to approximate
the relative dollar amounts of housing choice vouchers in each area. To
do so, we multiplied the FMR (FY 2004) and the number of vouchers for
each FMR area over 12 months.
[End of figure]
Because data developed from a single year of ACS data will be based on
samples that are approximately one-sixth as large as decennial census
long-form samples, HUD may need more data points than what the ACS will
provide for communities with smaller populations in order to estimate
FMRs. More specifically, according to HUD officials, to obtain a
sufficient sample of rent data for HUD's program purposes, the agency
needs data from areas with larger populations--that is, areas that can
provide more data points--than the ACS will publicly report. For
instance, in an annual ACS sample from a metropolitan area with a
population of 100,000, HUD could expect to find in ACS data only 48
recent movers in two-bedroom rental units, but it needs 200 recent
movers for its purposes. In order for HUD to obtain its needed minimum
sample of 200 units, it will likely need to use 1-year average data for
counties with populations of more than 400,000; 3-year average county-
level data for areas with populations of 133,000 to 400,000; and 5-year
average county-level data for areas with populations of less than
133,000.
In addition, although the Census Bureau will publish 3-and 5-year
rolling average ACS data every year beginning in 2008 and 2011,
respectively, HUD may not use these data every year because of concerns
about their reliability for HUD's FMR estimation purposes. According to
the Census Bureau, reliable measures of changes in multiyear averages-
-such as what HUD needs in order to estimate FMRs--should only be
calculated using averages with no overlapping years. The 3-and 5-year
rolling average ACS data that the Census Bureau publishes every year
will have overlapping years. For example, in 2008, the Census Bureau
will publish 3-year average ACS data covering 2005, 2006, and 2007; in
2009, it will publish 3-year average ACS data for 2006, 2007, and 2008,
overlapping the previous year's estimate by including 2006 and 2007
data. For HUD's purposes, a reliable time series of 3-year averages
would consist of the ACS data that the Census Bureau will publish in
2008 (2005-07 averages), 2011 (2008-10 averages), 2014 (2011-13
averages), and so on because these would not have overlapping years.
ACS Data Pose Certain Challenges to HUD That May Affect FMR Estimation
and Accuracy:
HUD's consultant ORC Macro, NAS, the Census Bureau, and we have
identified certain challenges associated with using ACS data that may
affect how and when HUD could use the data and improve the accuracy of
FMRs. The challenges include issues related to the averaging of the ACS
data, presentation of inflation-adjusted costs (such as rents),
techniques to deal with missing responses, and reporting differences
between the decennial census and the ACS.
Averaging:
The Census Bureau collects data for the ACS monthly and continuously
averages them over 1-, 3-, and 5-year time periods. However, this
averaging could hide rental market shifts because moving averages tend
to "smooth" changes in data over time.[Footnote 26] For example, if
from January through September of a given year the rent for an area is
$800, and from October through December of the same year the rent is
$1,200, the average annual rent reported by the ACS would be $900,
which is far less than the current monthly rent of $1,200. As a result,
the moving averages' "smoothing" effect may hide a turning point, or,
current prices in the rental housing market.
Inflation-Adjusted Costs:
To adjust for general inflation, the Census Bureau will use a general
adjustment factor rather than an index that is specifically related to
data items, such as rents or utilities, to present dollar-denominated
data from the ACS. This could limit the usefulness of the data for
HUD's program purposes because using a general adjustment factor (i.e.,
national CPI) rather than using an index that is specifically related
to the dollar-denominated item (i.e., a rent index) could result in a
less-precise estimate.[Footnote 27] The treatment of dollar-
denominated data is critical to all users of these data, and
particularly to HUD, which will be using the ACS to determine FMRs
based on rent data. If HUD had access to the Census Bureau's unadjusted
annual data, it could then adjust the data pertinent to its FMR
estimation using rent or utility indexes. We previously raised concerns
about the Census Bureau inflation adjustment.[Footnote 28] In response,
the Census Bureau did not provide a rationale for using the general
adjustment factor, rather than a more specific index, but did indicate
that the bureau would reconsider its present policy of showing only the
inflation-adjusted annual estimates.
Techniques to Deal with Missing Responses:
A NAS panel and we have raised concerns about how imputation--a
technique used to deal with surveys with missing responses--could
affect the accuracy of ACS data, especially in smaller areas. The NAS
panel that reviewed the 2000 Census raised issues about the potential
effects of imputation on ACS results. Unlike the process used for the
decennial census--100 percent follow-up for all nonrespondents--the
Census Bureau conducts follow-up on only 33 percent of nonrespondents
to the ACS. The Census Bureau uses the responses from the follow-up
surveys to attribute a similar pattern of responses to the remaining 66
percent of nonrespondents. The NAS panel called on the Census Bureau to
analyze the associated trade-offs in costs and accuracy between
imputation and additional fieldwork to gather more data.[Footnote 29]
Reporting Differences between the Decennial Census Long Form and the
ACS:
In a 2004 study, the Census Bureau found that when the decennial census
long form and the ACS were used to survey the same area, they reported
a number of variables differently, including those HUD uses to estimate
FMRs.[Footnote 30] The variables they reported differently include
housing occupancy, the year the structure was built, the number of
rooms, and gross rent. For instance, the study found that for certain
areas, the ACS reported moderately lower gross rents than did the
decennial census. According to the Census Bureau, the differences may
result partly from different survey processing techniques or from the
multiyear aspect of ACS data. Regardless of the cause, FMRs for fiscal
year 2008 (the first year of rebenchmarking with ACS data) could show
bigger changes than would be the case using decennial census data.
According to HUD, consistent FMRs--that is, estimates that change
gradually from year to year--are important because wide year-to-year
fluctuations, especially those changes that lower the FMR, can be
disruptive to PHAs, which must annually reconsider their payment
standards any time HUD changes the FMR.
HUD will address the ACS challenges when it receives and begins to
analyze 2005 ACS data--that is, the data collected during the first
year when the ACS is fully implemented--in Fall 2006. HUD may choose to
participate in an ACS Technical Workshop led by the Census Bureau,
which may help the agency address these challenges.
Despite Challenges, the ACS Remains Likely the Best Data Source for
FMRs:
Despite the challenges the ACS poses for HUD, neither we nor various
researchers and industry experts found reason to suggest (1) that HUD
should not go forward with its plans to use the ACS or (2) that there
are viable alternatives to the ACS. Other sources of information, such
as private-market rent data and tax assessment data, typically do not
contain the information that HUD needs to estimate FMRs. For example:
* Private-market rent data typically include more expensive properties
(i.e., luxury units, usually large apartment complexes, in metropolitan
areas). Most voucher holders do not rent such properties because they
cannot afford them. Additionally, these data do not include single-
family homes--properties that voucher holders may also lease.
* Private-market and tax assessment data are typically of lesser
quality compared with the data sources that HUD generally uses to
estimate FMRs. Private-market rent data often do not contain a
representative sample of the full rent distribution in an FMR area.
* Private-market or tax assessment surveys that include rent data may
not consistently include questions that ensure the units included
adhere to HUD's criteria (e.g., rents only from recent movers).
HUD Did Not Follow One of Its Data Quality Guidelines and May Lack Data
Sources to Assess the Accuracy of Future FMRs:
The potential exists for HUD to improve how it estimates FMRs and their
accuracy because (1) the agency presently does not follow its
objectivity guideline for ensuring the transparency and reproducibility
of its data and methods for estimating its FMRs and (2) it may in the
future lack a way to assess the accuracy of ACS-based rent estimates
when other information, such as comments from public housing agencies,
suggests it may need to do so. Various federal agencies, including HUD,
have developed guidelines to ensure they disseminate quality data.
Three of HUD's standards--utility, integrity, and objectivity--apply to
FMR estimation. Although HUD appears to be following the utility and
integrity guidelines, it did not follow its objectivity guideline--
which calls for the agency to make its data sources and methods
transparent so the results can be independently reproduced.
Additionally, as HUD comes to depend less on RDD survey and AHS data,
it may not have a means to assess the accuracy of future FMR estimates.
HUD Has Not Followed Its Data Quality Guideline on Objectivity:
Section 515 of the Treasury and General Government Appropriations Act
for Fiscal Year 2001 (Pub. L. No. 106-554) directs OMB to issue
governmentwide guidelines that provide policy and procedural guidance
to federal agencies for ensuring and maximizing the quality,
objectivity, utility, and integrity of information disseminated by the
agencies. According to OMB, information that has been subject to
independent reanalysis is generally presumed to be of acceptable
objectivity and therefore reliable to the user. In addition, OMB states
that an important benefit of transparency and reproducibility
(objectivity) is that the public can assess how much an agency's
information hinges on the specific analytical choices of the agency. In
response to OMB's guidelines, various federal agencies, including HUD,
have developed similar guidelines for ensuring that they disseminate
quality information. HUD's guidelines include ensuring the utility
(usefulness), integrity (protection from unauthorized access), and
objectivity (transparency and reproducibility) of the data it
disseminates.
Based on our review of available information, HUD appears to be
following the utility and integrity components of its guidelines for
FMRs. HUD's utility guideline states that the information disseminated
should be useful--a standard that encompasses accessibility and
timeliness. HUD follows this guideline by estimating FMRs on an annual
schedule and making FMRs public and easily accessible by publishing
them on its Web site and in the Federal Register. HUD's integrity
guideline states that the information disseminated should be protected
from corruption or falsification by unauthorized access or revision.
According to HUD officials, FMR data are kept on an internal server
with highly restricted access. Furthermore, to ensure the security of
the system, the officials said they maintain full electronic backups of
all systems.
However, we found that HUD does not follow its guideline pertaining to
objectivity. HUD's guidelines state that it will make publicly
available the sources, data, and methods used to develop the
information it disseminates, and that results must be capable of being
"substantially reproduced." This means that independent reanalysis of
original or supporting data using the same methods should generate
similar analytical results. Although HUD generally describes its
overall methodology for estimating FMRs in publicly available
documents, the agency has not documented its methodology in sufficient
detail to permit the results to be independently reproduced. For
example, although we obtained information on the data and methods HUD
used to estimate FMRs for fiscal years 2000-05, HUD's process was not
sufficiently documented to allow us to reproduce FMRs without
contacting HUD staff to assist us in doing so. In part, this was
because some of the data HUD used to estimate FMRs, such as utility
cost data, no longer exist after the agency upgraded the software it
uses to develop FMRs. Also, HUD did not document some of the key
procedures, variables, and data it used in estimating FMRs, such as the
source of benchmarking data (and its rationale for choosing each source
in any given year).[Footnote 31] Sufficient documentation would have
allowed outside parties to understand and assess how HUD developed any
given FMR. For example, sufficient documentation would allow an outside
party to determine (1) every decision HUD made (such as the FMR area
definition or survey source), (2) the decision rules it applied in
making that decision, and (3) the extent to which HUD consistently
applied these rules.
HUD's Declining Use of RDD Surveys and AHS Data May Limit Its Options
for Assessing the Accuracy of Future FMRs:
HUD officials state that they do not have a plan to assess the accuracy
of FMRs after they start using ACS data to estimate them, in part
because they believe they will no longer have a quality comparison
point or data with which to do so. In the past, HUD assessed accuracy
by comparing FMR estimates with the rents derived from a subsequent RDD
survey, the AHS, or a decennial census. However, HUD plans to limit its
future use of RDD surveys and the AHS because of their concerns about
cost and quality. According to HUD officials, RDD surveys are very
expensive (costing upwards of $20,000) and their reliability is
decreasing. Currently, according to HUD officials, the agency has to
start with a sample of 97,000 units to obtain a usable sample of 200
with which to estimate FMRs. Moreover, the response rate for RDD
surveys is about 40 percent, compared with 90 percent for the ACS, and
RDD surveys may have nonresponse bias (i.e., people who respond to
surveys may answer questions differently than those who do not).
Similarly, the AHS is becoming less useful for HUD's purposes than when
that survey first began. According to HUD officials, the number and
sample sizes of AHS metropolitan area surveys has been decreasing over
the past two decades, and they are not timely for HUD's program
purposes, thereby making them less useful for estimating FMRs than has
been the case in the past. Rent data from other sources, such as
private-market rent surveys and tax assessment records, also would not
provide HUD with a usable comparison point with which to assess FMR
accuracy.[Footnote 32]
Nonetheless, HUD's regulations require that the agency allow the public
to provide comments on proposed FMRs, and its information quality
guidelines permit affected parties to seek and obtain correction of
information disseminated by the agency. This extends to the accuracy of
FMRs. In addition to what its policies may require, even though FMRs
based on ACS data will most likely be more accurate than previous FMRs,
HUD officials acknowledge that ACS-based FMR estimates may be
inaccurate from time to time. For example, FMRs for the smaller areas
(rebenchmarked every 3 or 5 years with ACS data) may need to be
assessed within the interval to ensure that they remain accurate
between rebenchmarkings. Moreover, FMRs for any areas with volatile
rental markets may need to be assessed with some frequency to ensure
that they are accurate. However, as previously noted, HUD may lack
sources of comparable data in the future and may be unable to perform
these assessments.
Conclusions:
HUD's task in accurately estimating FMRs is formidable. It must produce
estimates for hundreds of areas throughout the country despite having
few comprehensive, reliable data sources with which to do so.
Additionally, HUD faces the normal difficulties associated with
predicting how rents and housing markets will change months (or years)
into the future. Nonetheless, for those affected by FMRs, such as
voucher holders, HUD's ability to produce accurate estimates each and
every year is vital--estimates that are too low make it more difficult
for low-income households to find housing they can rent with a voucher,
while estimates that are too high may needlessly waste resources or
prevent housing agencies from serving more households.
At the time of our review, HUD could not dispel concerns about its
process for estimating FMRs because its methodology is not transparent
enough to allow others--including GAO--to independently analyze its
rent data and produce similar results. HUD and those who use FMRs would
benefit from a more transparent methodology because this could enhance
the credibility of the estimates by clearly delineating the choices HUD
makes, what alternatives it may have had in making those choices, and
the decision rules it applied; for example, whether to use OMB's area
definitions or how much to modify them (and the basis for doing so).
Making the methodology transparent would also give users more and
better information with which to consider whether FMRs reliably reflect
an accurate estimate of the rents voucher holders and others will
encounter.
The advent of a new data source holds promise for HUD because a system
of FMRs that are largely based on the ACS will likely improve the
quality and accuracy of these estimates. However, the level of
improvement in the quality and accuracy of FMR estimates depends on how
HUD uses the ACS data. By choosing to use regional-level data to update
fiscal year 2006 FMRs rather than the more local state-level data, HUD
may not be taking full advantage of the new data source as soon as it
can.
In addition, as it transitions to the ACS, HUD expects to discontinue
its use of other surveys like the RDD surveys and the AHS to assess the
accuracy of its FMRs and, therefore, will not have a means to assure
itself and others that any given FMR estimate is accurate, particularly
when it receives public comments or other information suggesting it
needs to do so. While we agree that HUD is right to be concerned about
the escalating costs and declining quality of surveys such as the RDD
surveys, having no reasonable alternative to assess the accuracy of an
FMR will not likely address the concerns of PHAs with reason to
question FMR accuracy and may also contradict HUD's own data quality
guidelines.
Recommendations for Executive Action:
To improve the usefulness of its FMR estimates, we recommend that the
Secretary of HUD take the following three steps:
* ensure that HUD fully documents its method for estimating FMRs by
following all of its data dissemination quality guidelines,
particularly those pertaining to the transparency and reproducibility
of its methodology;
* use, as much as possible, the ACS data that corresponds more closely
to FMR areas to update the fiscal year 2006 FMRs; and:
* develop a mechanism to assess the accuracy of future FMRs, including
those that are based on the ACS, in instances where HUD learns of
information suggesting it needs to do so.
Agency Comments and Our Evaluation:
We provided a draft of this report to HUD for its review and comment.
In a letter from the Assistant Secretary for Policy Development and
Research (see app. II), HUD described our report as a good summary of
the intent of FMR estimates and the implementation of its methods. HUD
also suggested certain changes and clarifications to our report. For
example:
* HUD suggested that we present population-weighted accuracy estimates
in the "Highlights" page of our report. We agree that population-
weighted estimates are important and note that we present the
information in the body of the report rather than the "Highlights"
page.
* HUD provided a revised statement describing the process they use to
eliminate subsidized and nonstandard housing units from the rent
distribution. As HUD requested, we incorporated the new language in
footnote 16.
HUD agreed with our recommendation that it can better document its
methods for estimating FMRs, but also requested that we clarify certain
transparency and reproducibility issues in our report and recognize its
ongoing efforts in this regard. Among other things:
* HUD noted a distinction between process transparency and
reproducibility of results, stating that the public's needs are better
met by providing an overview of how FMRs are calculated and then
showing the individual calculations for each FMR estimate, rather than
providing system technical documentation, such as computer programs and
input data.
* HUD has sought to make the data and calculation process publicly
available and transparent. For example, HUD noted that it currently
posts on its Web site publicly releasable versions of 2000 decennial
census detailed rent distribution files; FMR history files from the
Federal Register, including Annual Adjustment Factors; and a summary of
the general methodology and major data sources it uses to estimate
FMRs.
* HUD stated that it provided us with additional information, such as
computer programs and input data, it used to estimate FMRs, and met
with us as needed to explain the FMR methodology, including the large
number of different data sources, decision rules, complex decision
trees, and complex series of computer programs it uses to estimate
FMRs.
We agree that providing step-by-step calculation details for each FMR
estimate would contribute to process transparency. Moreover, we agree
that HUD currently makes the major data sources and general methodology
it uses to estimate FMRs publicly available on its Web site. However,
as our draft report noted, the current information that HUD makes
publicly available does not show the individual calculations for each
FMR estimate and therefore is not sufficient to substantially reproduce
FMRs, a standard set out in HUD's data quality guidelines.
With respect to reproducibility, in reviewing HUD's process for
estimating FMRs, we asked for and HUD provided additional information,
such as computer programs, input data, and associated documentation.
Because HUD did not have and could not provide us with critical
documents, such as a clear step-by-step guide or data dictionary, HUD
officials met with us to explain the various computer programs and
variables they used--a step that should not be necessary if the
objective is for us to be able to independently substantially reproduce
FMR estimates. Nonetheless, the information and explanations HUD
provided were not sufficient to allow us to independently reproduce FMR
estimates. As HUD noted in its comments, documentation of computer
programs and input data, such as it provided us, are not as useful as
step-by-step guidelines that clearly detail how it produces each FMR.
As a result, HUD indicated it plans to consolidate in one place all of
the information it uses to estimate FMRs and create a new tool, for
release in April 2005, detailing how it develops each FMR. By making
this information publicly available on its Web site, HUD expects to
improve the transparency and reproducibility of its FMR estimates,
particularly for the users of these estimates.
HUD disagreed with our recommendation that it use, as much as possible,
the ACS data that corresponds more close to FMR areas to update its
fiscal year 2006 FMRs. HUD indicated that many of the annual state-
level rent numbers have a pattern of erratic changes. However,
according to the Census Bureau, for states with populations of 1
million or more, annual ACS changes for 2001 to 2004 are generally
reliable. More importantly, as HUD officials indicated to us during our
review, a necessary first step in using these data to update fiscal
year 2006 FMRs would be to assess for each state whether anomalies or
other concerns might indicate a need to defer in certain instances
using the state-level ACS data. Accordingly, our recommendation was for
HUD to use the state-level data as much as possible, recognizing that
the agency could do so only in instances where the ACS data are
sufficiently reliable for this purpose, and we have retained the
recommendation.
HUD did not explicitly agree or disagree with our recommendation that
it develop a mechanism to assess the accuracy of future FMR estimates.
However, HUD disagreed with our draft report's statement that declining
use of RDD surveys and the AHS may limit its options for assessing FMR
accuracy. Specifically, HUD stated that even though the ACS will be
much more accurate than any other survey unless the other survey offers
more current estimates, ACS rent estimates will always lag by at least
a year (from the midpoint of the survey estimate); thus, use of
national rent data to trend the FMR estimate could lead to estimation
errors in housing markets with unusual rent increases or decreases.
Accordingly, HUD noted that one of the major challenges posed by the
ACS is how to identify those areas where the use of regional or
national trending factors results in estimation inaccuracy, and stated
that it is currently exploring two alternatives to deal with the
issues. Thus, although HUD stated that it disagreed with our statement,
the actions that it intends to take are consistent with our
recommendation.
HUD also suggested technical clarifications to our report, which we
have incorporated as appropriate.
As arranged with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after the date of this letter. At that time, we will send copies to the
appropriate congressional committees and to the Secretary of Housing
and Urban Development. We will also make copies available to others
upon request. In addition, this report will be available at no charge
on the GAO Web site at [Hyperlink, http://www.gao.gov].
If you have any questions about this report, please contact me at (202)
512-6878 or [Hyperlink, woodd@gao.gov] or Bill MacBlane, Assistant
Director, at (202) 512-6764 or [Hyperlink, macblanew@gao.gov]. Key
contributors to this report are listed in appendix III.
Sincerely yours,
Signed by:
David G. Wood:
Director, Financial Markets and Community Investment:
[End of section]
Appendixes:
Appendix I: Objectives, Scope, and Methodology:
To describe how the Department of Housing and Urban Development (HUD)
estimates fair market rents (FMR), we first analyzed statutes and HUD
regulations, reviewed HUD documents, and interviewed HUD officials to
identify each step that HUD takes to estimate FMRs, including the role
that the public has in the process. Further, we also spoke with nine
HUD field economists for each HUD region--typically, the first point of
contact for the public--to further understand the role that the public
can play in adjusting the FMR estimate.[Footnote 33] To identify and
describe the relevant characteristics of the major data sources HUD
uses to estimate FMRs, we reviewed agency documents.
To determine how accurate FMRs were, we compared two-bedroom FMRs that
HUD had in place for fiscal year 2000--that is, estimates derived from
HUD's revisions to its baselines and from its update processes--with
the results of the 2000 census.[Footnote 34] In addition, we compared
two-bedroom FMRs that HUD estimated for fiscal years 2001-05 with data
from surveys HUD and others conducted for 153 FMR areas over this
period. We assessed accuracy by way of a comparison to the decennial
census or other surveys because our own methodological experts as well
as others conducting similar research on these issues determined that
such a comparison is the best way to do so when the true values--that
is, the distribution of all rents--cannot be known. In conducting both
of these comparisons, we focused on two-bedroom units because HUD
directly estimates FMRs for these units from the decennial census and
its other rebenchmarking surveys. HUD does not directly estimate FMRs
for other bedroom sizes, making it not possible to do a comparison of
those FMRs to the results of a survey such as the American Housing
Survey (AHS) or a Random Digit Dialing (RDD) survey.[Footnote 35]
We performed an associative analysis to determine what components of
HUD's FMR estimation process may have explained the results we found
when we assessed accuracy (e.g., whether the estimate was for a
metropolitan or nonmetropolitan area).[Footnote 36] Our analysis was
limited to making associations between the components of HUD's
methodology and the accuracy of its FMR estimates; it did not allow us
to make a direct causal link between the two because all of the
information we needed was either no longer available or may not be able
to be captured by HUD's method for making these estimates.
Specifically, (1) HUD could not provide all of the data used to
estimate FMRs from 1990 to 2005, such as utility cost data, because
these were kept on individual staff's computers and in many cases were
not transferred when HUD moved its FMR data systems to a more advanced
server; (2) the lack of transparency we found relative to HUD's
objectivity guideline for data quality meant that we could not identify
and isolate specific components of its methodology to attempt a causal
(rather than associative) analysis; and (3) neither we nor HUD can
control for factors outside of HUD's estimation process that may affect
accuracy, such as sudden employment changes that cause an area's rents
to increase rapidly.
We present our analysis of the accuracy of FMR estimates in terms of
the degree (percentage) to which the FMR matched or was close to the
corresponding survey. For example, for the corresponding fiscal year
2000 FMRs and census data, we calculated the following for each FMR:
Survey (census)-Fair Market Rent Estimate = x percent Survey
(census):
This calculation produced a percentage that, in this example, we
characterize as the estimate being within x percent of the census. For
descriptive purposes, we arrayed these comparisons in increments of 10
percent because, in terms of the initial FMR, this is the range (90 to
110 percent of the FMR) in which the public housing agencies may set
their payment standards without prior approval from HUD.
To determine how and when the incorporation of the American Community
Survey (ACS) data might affect the accuracy of the FMR estimates, we
reviewed agency documents and interviewed HUD officials to determine
how the agency plans to use ACS data to estimate FMRs. We also analyzed
Bureau of the Census documents to compare characteristics of the ACS
data with those of the data sources HUD currently uses (the decennial
census long form, the AHS, and RDD surveys) to estimate FMRs.
Additionally, we reviewed research by the National Academy of Sciences
and ORC Macro, in addition to our own, on the use of ACS data.
To identify changes HUD could make to improve the way it estimates FMRs
and their accuracy, we first assessed HUD's process for estimating FMRs
against its data quality guidelines. More specifically, we analyzed
each HUD guideline--utility, integrity, and objectivity--and compared
them with HUD's method for estimating FMRs. We also interviewed HUD
officials to determine how the guidelines related to FMRs.
Additionally, on the basis of our analysis of the data characteristics
we found to be associated with greater accuracy in FMRs (recent, higher
quality, and more local), we interviewed housing industry experts that
either routinely work with housing data or are familiar with HUD's data
needs to identify potential alternative data sources that HUD could use
to estimate FMRs. We also interviewed HUD officials to determine the
availability and merits of alternative data sources.
We conducted our work in Washington, D.C., between May 2004 and
February 2005 in accordance with generally accepted government auditing
standards.
[End of section]
Appendix II: Comments from the Department of Housing and Urban
Development:
U.S. DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT:
ASSISTANT SECRETARY FOR POLICY DEVELOPMENT AND RESEARCH:
WASHINGTON, DC 20410-6000:
March 18, 2005:
Mr. David G. Wood:
Director, Financial Markets and Community Investment:
U.S. Government Accounting Office:
441 G Street, NW, Room 2440:
Washington, DC 20548:
Dear Mr. Wood:
Thank you for the opportunity to comment on GAO's draft report on
Section 8 Fair Market Rents, GAO-05-342. GAO's review of HUD's Section
8 Fair Market Rent (FMR) system provides a good summary of the intent
and implementation of the system. There are some items, however, for
which clarifications or changes are warranted. There are also some
points with which we disagree for reasons noted.
The major points where clarifications or changes are requested are as
follows:
1. The title of the "Highlights" report is inconsistent with the
content. The current title suggests that FMR estimates are inaccurate
but, as noted below and in the body of GAO's report, this is not the
case. A more accurate summary of the report would be that "HUD FMRs are
Generally Accurate And Should Improve With ACS Data, But Calculations
Should Be Made More Transparent."
2. The unweighted FMR area accuracy measure used in the "Highlights"
section is misleading. The accuracy percentages reported treat all 356
metropolitan and 2,303 non-metropolitan areas as equals. Loving County,
Texas, with a population of 67 is treated as the equal of Los Angeles
with its 9.5 million people. The most relevant accuracy standard would
be a voucher-weighted percentage, followed by a population-weighted
measure. Population-weighted accuracy percentages are provided in the
body of the paper, but the initial and misleading impression in the
"Highlights" section will receive more attention. Attachment 1 provides
data on all three measures.
The voucher-weighted accuracy results of 91 percent overall and 95
percent for metro areas should be referenced in the summary. It is
requested that the second paragraph of the "Highlights" section be
replaced as follows:
"The 2000 Census rent data needed to measure the accuracy of FMRs
became available four years after the Fiscal Year 2000 FMRs were
published, and provide the most reliable available accuracy reference
standard. Ninety-one percent of all Section 8 vouchers in 2000 were in
areas with FMRs that were within 10 percent of FMRs calculated using
the 2000 decennial Census. Ninety-five percent of metropolitan area
vouchers were in areas that met this standard. An equivalent accuracy
measurement is not available for 1990, but on a population-weighted
basis 73 percent of all FMRs were within 10 percent of 1990 Census-
based FMRs and 94 percent were within 20 percent. In 2000, 88 percent
of all population-weighted FMRs were within 10 percent of 2000 Census-
based FMRs and 98 percent were within 20 percent. This is a significant
improvement over FMR accuracy in 1990. Given that areas considered
likely to have FMR accuracy problems are given first priority for FMR
surveys, it is also indicative that about 73 percent of the 153 areas
surveyed since the 2000 Census had published FMRs within 10 percent of
the survey estimates and that many of the areas where concerns were
raised were found to have accurate FMRs."
3. FMR transparency and reproducibility issues raised in GAO's stud
require clarification and recognition of HUD efforts in these areas.
There are two approaches that could be taken to making FMRs
reproducible by interested members of the public. One is to provide all
of the data and computer programs used plus a detailed explanation of
system functioning and decision rules. There are statutory restrictions
on release of some of these data, but HUD has had Census produce
publicly releasable, close approximations of the data actually used.
The major deficiency of a systems documentation-based approach is that,
while any given FMR was developed with a limited number of
calculations, there are a sufficiently large number of different data
sources and decision rules to require complex decision trees and a
complex series of computer programs. Although HUD is willing to provide
all system documentation and computer programs, and did so for GAO's
review, its experience with GAO confirms its belief that the public's
needs are better met by providing an overview of how FMRs are
calculated and then showing the individual calculations for each FMR
area.
HUD is in agreement that additional efforts to improve transparency and
reproducibility are desirable, but requests that GAO's report be
modified in recognition of the following:
a. There is a significant difference between transparency and system
reproducibility. HUD has increasingly sought to make the data and
calculation process as publicly available and transparent as possible.
Its approach has been to make as much of the underlying data as
possible available and to provide a step-by-step guideline to show how
local FMRs were produced. Doing this is much easier than trying to
provide understandable versions of the computer programs and processing
stream used to produce FMRs. Documenting the FMR system to meet OMB and
HUD's Office of Information Technology documentation standards, as was
done in 2002, was both expensive and staff-intensive. HUD agrees that
this latter type of documentation is not useful to a typical interested
party, and believes that providing step-by-step calculation details for
each FMR area is more useful and provides more estimation transparency
than trying to expand on system technical documentation.
b. HUD has actively sought to make the FMR process more transparent and
reproducible. Examples follow:
i. HUD had the Census Bureau prepare special, publicly releasable
versions of the detailed rent distribution files used to calculate FMRs
and bedroom rent ratios as well as an explanation of the calculation
process and factors used. These have been placed on the HUDUSER website
with file documentation.
ii. HUD provides FMR history files, proposed and final Federal Register
FMR publications, FMR Annual Adjustment Factors, and a summary of the
methodology and data sources on its HUDUSER website.
iii. HUD has developed an easy-to-use, Fiscal Year 2005 FMR replication
tool that permits users to select an area of interest and to see how
the area's 2000 Census FMRs and bedroom ratios were developed. It
tracks the year-by-year process to update 2000 Census estimates (or
more current survey estimates) to produce revised final Fiscal Year
2005 FMRs. The information provided includes the source and type of all
update factors used. Historical FMRs and related updating documentation
will be provided by a related reference tool, although the underlying
information is already posted on the HUDUSER website. The new reference
tools are still being tested and improved, but should be made available
to the public in April 2005. A test version is available for GAO use
at: http://www.buduser.org/datasets/fmr/computations.btm.
Recognition of these efforts in the GAO report is requested.
c. HUD provided GAO with full documentation for Fiscal Year 2005 FMRs.
This included all FMR calculation processes, all computer programs, all
input data, and the step-by-step process that updated Census 2000 rents
and produced Fiscal Year 2005 FMRs, including the decision rules for
selecting each of the FMR data sources used. HUD also volunteered to
answer any questions related to this process and spent many hours doing
so with GAO staff. Except for Census 2000 base rents (which are based
on embargoed data), Fiscal Year 2005 FMRs can be reproduced to the
dollar by following the documentation provided to GAO. HUD also offered
to show how any given FMR rebenchmarked since the 1990 Census was
derived, but no longer maintains the mainframe and PC-based mix of
computer systems needed to systematically replicate all estimates at
once.
d. HUD disagrees with the criticism on page _47 that its estimates are
not capable of being "substantially reproduced". The benchmark FMRs for
any given year are from the Census, the American Housing Survey, a
Random Digit Dialing survey, or a local survey. The data sources and
amounts for any rebenchmarked FMRs are specified in the Federal
Register and in the FMR history file available on the HUDUSER website.
The FMR Annual Adjustment Factors (AAFs) used to update FMRs each year
are published in the Federal Re ig ster and posted on the HUDUSER
website. FMR estimates for any area can be reproduced with these data,
although the consolidation of all relevant data in one place, as being
done in the system that will be shortly released to the public, will
make reproducibility easier for the public.
4. Footnote 16 is misleading because it implies that the Fiscal Year
2005 FMRs introduced a proxy to eliminate subsidized and nonstandard
units, whereas HUD has always had to use proxies. This topic is
relatively complex and is one on which HUD is seeking outside expert
review to provide additional perspectives. HUD uses the housing quality
measures available in the Census to eliminate substandard units from
the rent distributions used to calculate FMRs. Those measures are
limited, and HUD's objective is to reflect a more rigorous quality
standard equivalent to Section 8 housing quality standards. A
statistically process for doing this based on extensive research is
used that works well for most areas. Moreover, available data from
various Census Bureau studies suggest that serious housing quality
deficiencies affect a very small percentage of the inventory and have
little impact on most FMR estimates. Inclusion of income-based,
subsidized housing rent charges is a matter of more concern in
developing FMR estimates. Inclusion of public housing and HUD Section 8
housing assistance programs could distort FMR estimates, and 2000
Census data do not identify such assisted housing units. HUD examined
the adjustment approach used with the 1990 Census data to correct for
this bias. It also examined alternatives that took advantage of ACS and
HUD administrative data in conjunction with 2000 Census data, and
implemented an approach that has the effect of ensuring that a larger
number of rental units are removed from the bottom of the respective
local rent distributions prior to estimating FMRs than the number of
locally assisted housing units.
It is requested that the original footnote 16 be replaced with the more
accurate statement that follows:
"Prior to 2005, HUD used data on unit quality and assistance from the
American Housing Survey to generate a proxy for public, assisted and
substandard housing. This adjustment was constant over the nation and
did not vary by bedroom size. In the 2000 rebenchmarking, HUD employed
American Community Survey and HUD administrative data to calculate a
sub-standard housing adjustment that is tailored to region and bedroom
sizes. This new proxy allows for larger adjustments in areas with more
public and assisted housing units and higher housing quality issues.
HUD still uses information from RDD and AHS surveys to eliminate
subsidized and nonstandard units from survey data."
5. The discussion of the American Community Survey (ACS) is internally-
inconsistent and is likely to confuse readers. Pages 35 and 36 of the
GAO draft report state that the ACS is a higher quality survey than
others available except the Census, that it offers more current data,
and that "despite challenges in using the data, neither the GAO nor
experts and researchers who routinely work with housing data sources
identified viable alternatives to the ACS." Page 44 reiterates this
conclusion. The report then goes on to raise concerns about the ACS and
the lack of HUD efforts to develop alternatives to test the accuracy of
the ACS. In response, the following comments are offered:
a. GAO's concerns about the ACS accuracy apply almost equally to Census
data-the sample sizes and response rates are so much higher than from
any other source that evaluating the accuracy of the data using other
reference information is rarely feasible. The ACS is so much better
than other data sources that it will become the "gold" reference
standard once full annual samples are available. The ACS should provide
annual FMR estimates almost as reliable as those from the Census for
the largest metro areas and provide highly reliable estimates for all
other metro areas using either two or three years of data. HUD's
concern with the wording of the GAO report is that uninformed readers
may miss the point that the ACS is an impressive improvement over any
data from any source other than the decennial Census.
b. HUD disagrees with GAO's recommendation to use partial ACS sample
data at the state rather than the HUD Regional level in developing
Fiscal Year 2006 FMRs. The ACS sample was not fully implemented until
the start of 2005, and the previous year samples were much smaller and
highly clustered. In analyzing ACS data, we have found that many of the
annual state-level rent numbers show a pattern of erratic change
suspiciously similar to what one would expect from small samples. We
have no supporting evidence that instances where unusually high rent
changes are followed by unusually low rent changes (and vice versa)
reflect reality rather than sampling variability, which is why we
believe it is preferable to use multi-state regional rent change
factors until full sample ACS data for 2005 and subsequent years start
to become available.
6. HUD disagrees with the statement on page 48 that declining use of
RDD surveys and AHS data may limit its options for assessing the
accuracy of future FMRs. First, HUD anticipates continuing to review
AHS surveys and to make limited use of RDD surveys. HUD's underlying
disagreement with this statement, however, relates to the relative
accuracy of the ACS surveys. The ACS surveys will be so much more
accurate than AHS or RDD surveys for metropolitan areas that, as noted
in the GAO report on page 44, other available surveys are less reliable
and therefore cannot be used to evaluate the accuracy of fully
implemented ACS estimates. Other surveys are likely to be of value only
if they offer more current estimates. ACS survey data releases will
always lag by at least a year from the mid-point of the survey
estimate, and there will be longer lags for all but the largest areas.
Use of national rent trend information to cover the period from the ACS
survey to the as-of date of the FMR estimate will work well for the
large majority of areas, but lead to estimation errors in housing
markets with unusual rent increases or decreases. One of the major
challenges posed by ACS data is how to identify the relatively small
number of areas where the use of regional or national trending factors
to cover the lag period results in estimation accuracy problems. HUD is
currently exploring two alternatives to deal with this issue.
Thank you for your consideration of these comments. Please do not
hesitate to contact me at 202-708-1600 should you have any questions.
Sincerely,
Signed for:
Dennis C. Shea:
Attachment 1:
Published Versus Subsequently Available Census-Based FMRs:
[See PDF for image]
[End of table]
[End of section]
Appendix III: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
David G. Wood, (202) 512-6878;
Bill MacBlane, (202) 512-6764:
Staff Acknowledgments:
In addition to the individuals named above, Triana Bash, Tania Calhoun,
Steve Brown, John Larsen, Marc Molino, Chris Moriarity, Robert Parker,
MacDonald Phillips, Carl Ramirez, Barbara Roesmann, and Anita Visser
made key contributions to this report.
(250205):
FOOTNOTES
[1] As part of the decennial census since 1960, the Census Bureau has
mailed separate long-form questionnaires to a sample of households to
collect detailed information on demographic, housing, social, and
economic characteristics.
[2] When households rent units for less than the payment standard, the
HUD subsidy is the difference between their gross rent and their income
contribution.
[3] The Moderate Rehabilitation Single-Room Occupancy (SRO) program
provides rental assistance for homeless persons in connection with the
rehabilitation of SRO dwellings.
[4] HUD's Mark-to-Market Program reduces rents to market levels for
expiring housing subsidy contracts and restructures existing debt to
levels supportable by these rents on thousands of privately owned
multifamily properties with federally insured mortgages.
[5] HUD's HOME Program helps to expand the supply of decent, affordable
housing for low-and very-low-income families by providing grants to
states and local governments to fund housing programs that meet local
needs and priorities.
[6] HOPWA addresses the specific needs of persons living with HIV/AIDS
and their families by making grants to local communities, states, and
nonprofit organizations for purposes such as facility operations or
rental assistance.
[7] The LIHTC Program is an indirect federal subsidy used to increase
the supply of affordable housing in communities by financing the
development of affordable rental housing for low-income households.
Difficult development areas are designated by the Secretary of HUD as
areas that have high construction, land, and utility costs relative to
the area median gross income.
[8] 42 U.S.C. § 1437f(c)(1).
[9] See 24 C.F.R. § 888.113 for regulations governing the FMR
methodology.
[10] Beginning in 2001, HUD set FMRs for 39 metropolitan areas at the
50TH percentile, because it determined that an FMR increase was needed
to promote residential choice, help families move closer to areas of
job growth, and alleviate concentrations of poverty.
[11] Rents for units on 10 or more acres and seasonal units, such as
summer rentals, are ineligible for the FMR estimation process.
[12] The ACS is subject to annual appropriations. Funding for the ACS
to cover all persons except those living in group quarters (e.g.,
college dormitories and prisons) was approved beginning with fiscal
year 2005. Funding to cover all persons has been requested beginning
with 2006.
[13] The first annual ACS data for geographic areas with populations
larger than 65,000 will be published beginning in 2006; publication of
3-year averages for areas with populations of 20,000 to 65,000 will
begin in 2008; and publication of 5-year averages for areas with less
than 20,000 will begin in 2010.
[14] According to OMB, a metropolitan area generally consists of a core
area containing a substantial population nucleus, and adjacent
communities exhibiting a high degree of economic and social integration
with the core.
[15] In 1994, we reported on a proposal to establish smaller FMR areas.
See GAO, Rental Housing: Use of Smaller Market Areas to Set Rent
Subsidy Levels Has Drawbacks, RCED-94-112 (Washington, D.C.: June 24,
1994).
[16] Prior to 2005, HUD used information on unit quality and assistance
from the AHS to generate a proxy for subsidized (public and assisted)
and substandard housing. This adjustment was constant over the nation
and did not vary by bedroom size. To estimate fiscal year 2005 FMRs,
HUD used ACS and HUD administrative data to calculate a substandard
housing adjustment that is tailored to region and bedroom sizes.
Specifically, HUD began to use the 75TH percentile of public housing
rents from its administrative data for each of its regions as a proxy
to indicate which units are subsidized and substandard. According to
HUD, this new proxy allows for larger adjustments in areas with more
public and assisted housing units and higher housing quality issues.
HUD continues to use information from RDD surveys and the AHS to
eliminate subsidized and substandard units from survey data.
[17] The 2000 decennial census produced data sufficient to allow HUD to
directly estimate FMRs for all bedroom sizes for fiscal year 2005 FMRs
and update the bedroom ratios. According to HUD officials, they will
use these new ratios to estimate future non-two-bedroom FMRs.
[18] In very limited instances, HUD officials will accept data from
PHAs in areas with small populations that have not followed the
requirements. According to HUD officials, some areas with small
populations will not be able to comply due to limited budgets or small
sample sizes within the FMR area. HUD officials then evaluate the data
on the basis of their professional judgment.
[19] In order to obtain an exception to increase the payment standard
by more than 10 percent, the public must submit documentation that
demonstrates approval of the special exception is necessary to prevent
financial hardship for families in the exception area. This
documentation can include census rent data, locally funded quality
surveys, lease rates, and success rates. The request must be needed (1)
to enable families to find housing outside areas of high poverty and
(2) because voucher holders have trouble finding housing for lease.
[20] We use "associated with accuracy" because our analysis does not
enable us to make causal links between survey or FMR area
characteristics and the accuracy of estimates.
[21] Most households receiving tenant-based vouchers--85 percent--live
in metropolitan areas.
[22] Traditional surveys are surveys of rent data in metropolitan areas
with relatively low populations in which a PHA or other entities have
access to all or almost all of the rents in the area--for example, in
cities or towns that require owners to register rents annually and
maintain a database of rents. Telephone surveys are generally derived
from randomly selected lists of residential telephone numbers but are
not assisted by the use of a computer to track telephone calls and the
outcomes.
[23] According to HUD, some nonmetropolitan areas have unusually low
census data sample sizes and unusually high levels of substandard and
assisted housing that may distort the accuracy of FMR estimates. For
the fiscal years 1996 through 2004 FMRs, HUD corrected for low FMR
estimates that were at or below the cost of operating housing by
implementing a minimum FMR level for each state.
[24] According to HUD, a PHA utility schedule is a list of the average
monthly costs of various types of utilities, such as heating oil,
electricity, or water and sewer charges, subdivided by the number of
bedrooms in the unit.
[25] The Census Bureau reports that the 5-year averages will be about
as accurate as the long-form data; the annual and 3-year averages will
be significantly less reliable than the long-form data but more
reliable than existing household surveys the Census Bureau conducts.
[26] See ORC Macro, The American Community Survey: Challenges and
Opportunities for HUD (Calverton, MD: Sept. 27, 2002). ORC Macro is the
consultant HUD hired.
[27] See GAO, American Community Survey: Key Unresolved Issues, GAO-05-
82 (Washington, D.C.: Oct. 8, 2004).
[28] See GAO-05-82.
[29] See GAO-05-82.
[30] U.S. Census Bureau, Meeting 21ST Century Demographic Data Needs-
Implementing the American Community Survey Report 8: Comparison of the
American Community Survey Three-Year Averages and the Census Sample for
a Sample of Counties and Tracts (Washington, D.C.: 2004).
[31] For example, if the survey source for an FMR estimate was the AHS,
HUD's documentation did not indicate what other sources, if any, it
considered that year and why it chose the AHS over any other available
sources of data for that year.
[32] According to ORC Macro, HUD may be able to use special ACS
tabulations from the Census Bureau to detect shifts in rent trends for
areas where HUD will use multiyear average data to estimate FMRs. These
data for each FMR area may not contain enough samples to estimate FMRs,
but would give HUD an indication that an existing FMR may be
inaccurate.
[33] As of December 2004, nine HUD regional field economists managed
the agency's economic work in the 10 HUD regions because there was a
vacancy in Region 2 (New York/New Jersey).
[34] In order to use the 2000 decennial census data we obtained from
HUD to assess the accuracy of FMRs, we verified the reliability of the
census data by asking HUD officials a series of data reliability
questions.
[35] For non-two-bedroom units in the 2000 decennial census survey and
153 subsequent rebenchmarking surveys, HUD took the survey results for
two-bedroom rents and applied a rent ratio that, in HUD's view,
captured the approximate relationship between rents for two-bedroom
units and other sizes. For example, through fiscal year 2004, for three-
bedroom units, HUD determined that the relationship between these and
two-bedroom rents was 1.25, so the three-bedroom FMR would be 125
percent of what HUD estimated for two-bedroom units.
[36] When we compared the accuracy of FMR estimates with the two types
of update factors HUD uses (metro-specific Consumer Price Index or RDD
regional gross rent change factor), we excluded a limited number of FMR
areas because HUD applies special rules for updating this group, making
the update calculations too dissimilar for our purposes.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
