Public Housing

Small and Larger Agencies Have Similar Views on Many Recent Housing Reforms Gao ID: GAO-04-19 October 30, 2003

In response to long-standing concerns, HUD initiated efforts to improve the administration of its programs in 1997, and Congress passed the Quality Housing and Work Responsibility Act (QHWRA) in 1998. The act contains over 80 reforms that affect two key rental housing assistance programs: the low-rent housing program (also referred to as public housing) and the Housing Choice Voucher program (formerly Section 8). According to many housing agencies, implementing these reforms challenged their ability to address their core mission of providing safe, decent, and sanitary rental units for low-income residents. In particular, some small agencies that manage properties with relatively few rental units have contended that some reforms have little relevance to their operations and pose a significant burden because of the agencies' limited staff and financial resources. In response to a Congressional request, GAO compared housing agencies by size in terms of (1) the impact of recent housing reforms on their ability to administer HUD programs, (2) the agencies' performance as measured by HUD, and (3) the differences in the technical assistance that the agencies require. To carry out its work, GAO surveyed a statistical sample of small and larger public housing agencies nationwide on the impact of QHWRA reforms. The response rate to the survey was about 69 percent.

Of the 18 QHWRA reforms GAO asked about in its national survey, housing agencies of all sizes had similar views on 11 of them. However, when asked to what extent these reforms affected their operations, agencies' responses sometimes differed by size category. For example, the largest percentages of agencies in all size categories viewed the annual plan reform--which requires agencies to provide certain information pertaining to their upcoming fiscal year--as helpful to them in managing and operating their agencies. But small agencies indicated that this reform helped to a lesser extent than larger agencies. Other results of our survey: (1) all agencies reported spending more time on administering HUD programs since QHWRA was implemented, (2) most agencies said they contracted out about the same amount of property management and services as before QHWRA, and (3) about 75 percent of small agencies reported they believe the regulatory changes HUD has issued to reduce administrative burden will help. Performance ratings varied between small and larger housing agencies, according to HUD assessment data. The ratings, which assess the agencies' management of HUD housing programs, showed small agencies scoring better than larger ones in managing low-rent units. However, smaller agencies received lower scores than larger agencies for managing Housing Choice Voucher units, but this result may be due partly to HUD scoring method. HUD uses its risk assessment and management performance assessment systems to target assistance based on its determination of those that need it most. In addition, housing agencies also contact HUD directly to request assistance. However, according to HUD field office officials, small agencies are more likely to need assistance with day-to-day management issues than large agencies because small agencies tend to have few staff that specialize in key areas that are important to managing HUD's programs.



GAO-04-19, Public Housing: Small and Larger Agencies Have Similar Views on Many Recent Housing Reforms This is the accessible text file for GAO report number GAO-04-19 entitled 'Public Housing: Small and Larger Agencies Have Similar Views on Many Recent Housing Reforms' which was released on December 01, 2003. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Ranking Minority Member, Subcommittee on Housing and Transportation, Committee on Banking, Housing, and Urban Affairs, U.S. Senate: October 2003: PUBLIC HOUSING: Small and Larger Agencies Have Similar Views on Many Recent Housing Reforms: [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-19] GAO-04-19: GAO Highlights: Highlights of GAO-04-19, a report to the Ranking Minority Member, Subcommittee on Housing and Transportation, Committee on Banking, Housing and Urban Affairs, U.S. Senate Why GAO Did This Study: In response to long-standing concerns, HUD initiated efforts to improve the administration of its programs in 1997, and Congress passed the Quality Housing and Work Responsibility Act (QHWRA) in 1998. The act contains over 80 reforms that affect two key rental housing assistance programs: the low-rent housing program (also referred to as public housing) and the Housing Choice Voucher program (formerly Section 8). According to many housing agencies, implementing these reforms challenged their ability to address their core mission of providing safe, decent, and sanitary rental units for low-income residents. In particular, some small agencies that manage properties with relatively few rental units have contended that some reforms have little relevance to their operations and pose a significant burden because of the agencies‘ limited staff and financial resources. In response to the request of the Ranking Minority Member of the Subcommittee, GAO compared housing agencies by size in terms of (1) the impact of recent housing reforms on their ability to administer HUD programs, (2) the agencies‘ performance as measured by HUD, and (3) the differences in the technical assistance that the agencies require. To carry out its work, GAO surveyed a statistical sample of small and larger public housing agencies nationwide on the impact of QHWRA reforms. The response rate to the survey was about 69 percent. What GAO Found: Of the 18 QHWRA reforms GAO asked about in its national survey, housing agencies of all sizes had similar views on 11 of them. However, when asked to what extent these reforms affected their operations, agencies‘ responses sometimes differed by size category. For example, the largest percentages of agencies in all size categories viewed the annual plan reform”which requires agencies to provide certain information pertaining to their upcoming fiscal year” as helpful to them in managing and operating their agencies. But small agencies indicated that this reform helped to a lesser extent than larger agencies. Other results of our survey: * All agencies reported spending more time on administering HUD programs since QHWRA was implemented. * Most agencies said they contracted out about the same amount of property management and services as before QHWRA. * About 75 percent of small agencies reported they believe the regulatory changes HUD has issued to reduce administrative burden will help. Performance ratings varied between small and larger housing agencies, according to HUD assessment data. The ratings, which assess the agencies‘ management of HUD housing programs, showed small agencies scoring better than larger ones in managing low-rent units. However, smaller agencies received lower scores than larger agencies for managing Housing Choice Voucher units, but this result may be due partly to HUD scoring method. HUD uses its risk assessment and management performance assessment systems to target assistance based on its determination of those that need it most. In addition, housing agencies also contact HUD directly to request assistance. However, according to HUD field office officials, small agencies are more likely to need assistance with day- to-day management issues than large agencies because small agencies tend to have few staff that specialize in key areas that are important to managing HUD‘s programs. www.gao.gov/cgi-bin/getrpt?GAO-04-19. To view the full product, including the scope and methodology, click on the link above. For more information, contact David G. Wood, (202) 512-8678, or woodd@gao.gov. [End of section] Contents: Letter: Background: Results in Brief: With Some Exceptions, Small Agencies' Views of QHWRA Reforms Were Similar to Those of Larger Agencies: Scores from HUD's Performance Measurement and Risk Assessment Systems Show That Small and Larger Agencies' Performance Varies: HUD's Technical Assistance Is Based on Agency Needs, Which Sometimes Vary by Agency Size: Agency Comments: Appendixes: Appendix I: Scope and Methodology: Appendix II: Methodology for GAO's Survey of Local Housing Agencies on QHWRA Housing Reforms and Initiatives: The Study Population: The Sample Design: Developing the Survey: Administering the Survey: Nonsampling Error and Data Quality: Response Rates: Estimates: Sampling Error: Appendix III: Survey Results of Housing Agencies: Appendix IV: Summary of 18 Reforms Contained in GAO's Survey of Public Housing Agencies: Appendix V: GAO Contacts and Staff Acknowledgments: GAO Contacts: Staff Acknowledgments: Tables: Table 1: Small, Medium, and Large Public Housing Agencies and Their Inventory, Based on GAO's Definition for Size, Fiscal Year 2002: Table 2: Average SEMAP Scores for Seven Indicators for Which All Agencies Are Scored, by Agency Size (Fiscal Year 2002): Table 3: Survey Sample Size and Disposition: Figures: Figure 1: Survey Estimates on Effects of Seven QHWRA Reforms That Primarily Helped Agencies to Operate and Manage HUD Programs: Figure 2: Survey Estimates on Four QHWRA Reforms That Primarily Neither Helped Nor Hindered Agencies to Operate and Manage HUD Programs: Figure 3: Survey Estimates on Seven Reforms with Varied Impacts on Agency Ability to Operate and Manage HUD Programs: Figure 4: Survey Estimates of Time Agencies Spend Administering HUD Programs Since QHWRA Reforms: Figure 5: Survey Estimates of Extent to Which Agencies Reported Increased Reporting Requirements and Changes in Existing Requirements Contributed to Increased Time Spent on HUD Programs: Figure 6: Survey Estimates of Extent to Which Difficulty in Submitting Data on the Low-Rent Program and Accessing HUD's Computer Systems Contributed to Increased Time Spent on HUD Programs: Figure 7: Survey Estimates of Extent to Which Late and Unclear Guidance and Changes in Regulations Contributed to Increased Time Spent on HUD Programs: Figure 8: Survey Estimates of Extent to Which Lack of Resources for Hiring and Training New Staff and the Need to Train Staff Contributed to Increased Time Spent on HUD Programs: Figure 9: Survey Estimates of Extent to Which Agencies Report Contracting Out for Property Management and Services Since QHWRA: [See PDF for image] [End of figure] Figure 10: Survey Estimates of Extent to Which Agencies Believe HUD's Regulatory Relief Will Help: [See PDF for image] [End of figure] Figure 11: Overall PHAS Scores, by Agency Size (Fiscal Year 2002): [See PDF for image] [End of figure] Figure 12: Overall SEMAP Scores, by Agency Size (Fiscal Year 2002): [See PDF for image] [End of figure] Figure 13: Overall PIC Risk Assessment Scores, by Agency Size (Fiscal Year 2002): Abbreviations: HUD: Department of Housing and Urban Development: NAHRO: National Association of Housing and Redevelopment Officials: PHADA: Public Housing Authorities Directors Association: PHA: public housing agency: PHAS: Public Housing Assessment System: PIC: Public and Indian Housing Information Center: QHWRA: Quality Housing and Work Responsibility Act: REAC: Real Estate Assessment Center: SEMAP: Section Eight Management Assessment Program: TARC: Troubled Agency Recovery Center: Letter October 30, 2003: The Honorable Jack Reed: Ranking Minority Member: Subcommittee on Housing and Transportation: Committee on Banking, Housing, and Urban Affairs: United States Senate: Dear Senator Reed: Over 4,100 state, county, and municipal housing agencies administer federal housing programs--which were funded at nearly $24 billion in fiscal year 2003--on behalf of the Department of Housing and Urban Development (HUD). In response to long-standing concerns, HUD initiated efforts to improve the administration of its programs in 1997, and Congress passed the Quality Housing and Work Responsibility Act (QHWRA) [Footnote 1] in 1998. QHWRA contains over 80 reforms that affect two key rental housing assistance programs: the Low-Rent Public Housing Assistance Program (also referred to as low-rent or public housing) and the Housing Choice Voucher program (formerly referred to as Section 8).[Footnote 2] According to many housing agencies, implementing these reforms has challenged their ability to address their core mission of providing safe, decent, and sanitary rental units for low-income residents. In particular, some small agencies that manage properties with relatively few rental units have contended that some reforms have little relevance to their operations and pose a significant burden because of the agencies' limited staff and financial resources. HUD annually assesses the operating performance of the housing agencies that administer the two programs, using performance measurement systems that assign agencies a score for each program. HUD uses these scores and ratings from its risk assessment system to identify housing agencies that may be in need of monitoring and technical assistance. HUD's technical assistance--sometimes called capacity building-- involves activities such as training staff at local housing agencies on how to use HUD systems or comply with HUD's reporting requirements. As agreed with your office, we (1) compared how small and larger housing agencies view the impact of recent housing reforms on their ability to administer HUD programs, (2) compared small and larger agencies' performance, as measured by HUD, in administering the low- rent housing and Housing Choice Voucher programs, and (3) described the differences in the technical assistance that small and larger public housing agencies require. For the purposes of this report, we classified agencies based on the number of low-rent and Housing Choice Voucher units they administered. Small agencies administered fewer than 250 units; medium agencies administered between 250 and 1,249 units; and large agencies administered 1,250 units or more. To compare how small and larger housing agencies view the impact of recent housing reforms on their ability to administer HUD programs, we developed a survey that we mailed to a statistical sample of public housing agencies nationwide. Of the more than 80 QHWRA reforms, our survey focused on the 18 reforms that public housing industry associations consider to have had the most impact on agencies' operations. Based on the sample, we produced national estimates of housing agency directors' perceptions about the impact of QHWRA reforms and related issues and tested for size-response differences. [Footnote 3] We analyzed respondents' most frequently occurring answers to our survey questions to identify similarities and differences between small, medium, and large agencies. To compare small and larger agencies' performance in managing HUD's low-income housing programs, we analyzed data from HUD's performance measurement assessment systems. To determine differences in the technical assistance that small and larger agencies require, we analyzed data from HUD's performance measurement and risk assessment systems and interviewed HUD headquarters and field office officials. Appendix I contains additional information on our scope and methodology. Appendix II contains details on our survey methodology, and appendix III contains our survey results. Background: Under the United States Housing Act of 1937, as amended, Congress created the federal public housing program to assist communities in providing decent, safe, and sanitary dwellings for low-income families. Today, more than 4,100 public housing agencies, typically local agencies created under state law, develop and manage public housing units for low-income families. Over 3,100 agencies operate low-rent or a combination of low-rent and Housing Choice Voucher programs, and about 1,000 provide housing through the voucher program only. Agencies that participate in the low-rent program contract with HUD to provide housing to eligible low-income households; in return, the agencies receive both capital and operating fund grants. Under the Housing Choice Voucher program, eligible households live in rental units they locate in the private housing market. Participating housing agencies that administer such units contract with HUD to receive funds for rent subsidies for the participating households. Congress and HUD Have Initiated Public Housing Reforms: During the 1990s, the nation's public housing agencies gained broader latitude from HUD and Congress to establish their own policies in areas such as selecting tenants and setting rent levels. In 1996, Congress enacted legislative reforms that allowed public housing agencies to set minimum rents and to drop all mandatory federal preferences for admission on the basis of hardships such as homelessness. These reforms gave housing agencies greater control over their social and fiscal environment, enabling them to tailor their policies to local needs and conditions. In 1998, Congress enacted the Quality Housing and Work Responsibility Act of 1998 (QHWRA).[Footnote 4] This act, which extensively amended the United States Housing Act of 1937, embodied many of the 1996 reforms, allowing public housing agencies to exercise still more discretion over rents and admissions. For example, QHWRA increased managerial flexibility by, among other things, making HUD- provided capital and operating funds more fungible; allowing housing authorities to sell some units to residents; converting some public housing buildings to the voucher system; and developing mixed-income housing units in order to bring more working and upwardly mobile families into public housing. QHWRA also imposed new requirements on housing agencies, including, for example, mandatory reporting requirements in the form of a 5-year plan and annual operating plans. This report focuses on the 18 QHWRA reforms public housing industry groups believe have had the greatest impact on housing agencies' operations. These reforms are described more fully in appendix IV. Reforms concerning agencies' operations: * Fungibility of capital and operating funds: gives agencies greater flexibility to use HUD-provided capital and operating grants interchangeably, within certain limits. * Certificate/voucher merger: requires agencies to adjust administrative operations, where needed, in response to the merger of two former federal housing assistance programs. * Five-year plans: requires agencies to prepare comprehensive 5-year plans with goals and objectives that serve needs of low-income households. * Annual plans: requires agencies to prepare 1-year plans that include information on the agency's low-rent and Housing Choice Voucher programs and policies. * Physical inspections: requires agencies to annually inspect their low-rent housing units. Reforms concerning rent setting: * Minimum rents: allows agencies to charge public housing residents a monthly minimum rent of not more than $50. * Flat rents: allows households to choose to pay either a flat rent based on market rates or rent based on a percentage of their household income. * Rent burden limitation: requires agencies to subsidize initial voucher recipients so that their rent does not exceed 40 percent of household income. Reforms concerning resident selection/admission: * Federal tenant preferences: repeals mandate that agencies give admissions preference to certain categories of prospective tenants and permits agencies to establish their own preferences within certain limits. * Deconcentration: prohibits agencies from concentrating very low- income households in public housing projects or buildings within projects. * Income targeting: requires agencies to have a mix of household income levels in both low-rent housing and Housing Choice Voucher units. * Income disregard: requires agencies to exclude certain types of income when determining household eligibility for assistance. * Site-based waiting lists: permits agencies to allow public housing applicants to designate specific projects in which they wish to reside, subject to certain limits. Reforms concerning residents: * Pet policy: requires agencies to allow residents one or more common household pets. * Community service: requires agencies to ensure that most adult public housing residents perform monthly community service. * Resident board member: requires most agencies to have on their board of directors at least one member who is directly assisted by the agency. * Resident advisory board: requires agencies to consult with the board of residents in preparing annual and 5-year plans. * Resident surveys: requires HUD to obtain information on the involvement of public housing residents in the administration of public housing, including residents' satisfaction with their living conditions and services. In response to concerns that some QHWRA reforms are placing an undue burden on small housing agencies, HUD recently issued regulations designed to lessen their regulatory burden.[Footnote 5] The rule is designed to relieve eligible small agencies from administrative burden in two areas--the annual plan requirement and annual Public Housing Assessment System (PHAS) and Section Eight Management Assessment Program (SEMAP) assessments. For example, under the rule, agencies who operate fewer than 250 low-rent units are allowed to submit streamlined annual plans for their low-rent units. In addition, for small agencies that are assessed under PHAS and SEMAP, the rule reduces the frequency of these assessments from an annual basis to every other year. HUD Assists Public Housing Agencies and Evaluates Their Performance: To improve the administration of its low-income housing programs, HUD provides technical assistance either directly or indirectly to housing agencies that request assistance or have been identified by HUD as needing it. HUD sometimes uses the terms "technical assistance" and "capacity building" interchangeably, and the definitions overlap.[Footnote 6] Technical assistance programs can generally be defined as training designed to improve the performance or management of program recipients, such as one-on-one training in the use and implementation of HUD's management assessment systems. Capacity building can generally be defined as funding to strengthen the capacity or capability of program recipients or providers--typically housing or community development organizations. The overall goal of both technical assistance and capacity building is to enhance the delivery of HUD's housing and community development programs, and some assistance efforts incorporate elements of each. The technical assistance is delivered by staff from HUD's headquarters or one of its 43 field offices or by contractors. According to HUD officials, most technical assistance is delivered to agencies that request it directly from one of HUD's field offices. HUD also provides technical assistance through several programs that are administered primarily by headquarters, such as Hope VI and the Capital Fund.[Footnote 7] HUD assesses the performance of public housing agencies so that the Secretary of Housing can evaluate agencies' performance in all major areas of management operations. To this end, HUD uses two systems to assess the management performance of public housing agencies: * PHAS is designed to obtain an independent evaluation of housing agencies' overall performance in managing low-rent units, including the physical condition of the units, the soundness of agencies' financial operations, the effectiveness of their management operations, and the level of resident satisfaction with the services and living conditions. HUD's Real Estate Assessment Center (REAC) arranges independent physical inspections of HUD's low-rent properties, collects and analyzes data on the financial and physical condition of the agencies, and evaluates information from resident satisfaction surveys.[Footnote 8] * SEMAP measures the performance of the housing authorities that administer Housing Choice Voucher units. SEMAP uses 14 indicators that measure factors such as procedures for selecting tenants, income determinations, and inspections. HUD uses the Public and Indian Housing Information Center (PIC) system to generate a risk assessment of housing agencies. PIC risk assessment scores take into account PHAS and SEMAP scores as well as funding and compliance information and a number of qualitative factors--for example, audit findings, court actions, and tenant claims against agencies. HUD uses PHAS, SEMAP, and PIC scores to identify agencies that are having performance problems and determine the kind of technical assistance needed to correct deficiencies. As a result of these assessments, HUD may determine that a housing authority is "troubled"- -that is, experiencing especially severe difficulties in managing its housing programs. HUD's Troubled Agency Recovery Centers (TARC), located in Cleveland, Ohio, and Memphis, Tennessee, provide technical assistance to troubled housing agencies.[Footnote 9] HUD Has Various Ways of Defining the Size of a Small Agency: HUD has several ways of defining the size of small agencies. For example, in its recent small agency deregulation rule, HUD provided two definitions for a small agency: one for submitting streamlined annual plans and less frequent PHAS assessments, and another for receiving less frequent SEMAP assessments. In a 1999 report by the Office of Policy Development and Research, HUD defined a small housing agency as one with fewer than 500 low-rent units.[Footnote 10] And in another report, HUD defined a small agency as one with fewer than 1,000 Housing Choice Voucher units.[Footnote 11] However, when HUD's REAC assesses the financial performance of housing agencies it classifies small agencies as those with less than 250 combined low-rent and Housing Choice Voucher units. In carrying out our work, we classified agencies by size based on the total number of low-rent and Housing Choice Voucher units they administered. Regardless of the criteria used to define a small agency, most housing agencies are considered small. Table 1 shows the number and percentage of small, medium, and large agencies according to the definition of size that we used in this report. This table also indicates that while most agencies are small, they operate a relatively small proportion of all units. Table 1: Small, Medium, and Large Public Housing Agencies and Their Inventory, Based on GAO's Definition for Size, Fiscal Year 2002: Size: Small; Number of agencies: 2,438; Percentage of agencies: 58.2; Low-rent units: 136,023; Housing Choice Voucher units: 86,271; Number of units: 222,294; Percentage of units: 6.8. Size: Medium; Number of agencies: 1,269; Percentage of agencies: 30.3; Low-rent units: 255,749; Housing Choice Voucher units: 442,945; Number of units: 698,694; Percentage of units: 21.3. Size: Large; Number of agencies: 482; Percentage of agencies: 11.5; Low-rent units: 858,528; Housing Choice Voucher units: 1,499,580; Number of units: 2,358,108; Percentage of units: 71.9. Size: Total; Number of agencies: 4,189; Percentage of agencies: 100.0; Low-rent units: 1,250,300; Housing Choice Voucher units: 2,028,796; Number of units: 3,279,096; Percentage of units: 100.0. Source: GAO analysis of HUD data. [End of table] Results in Brief: With some exceptions, small agencies' views on the impact of QHWRA reforms on their ability to administer HUD programs were similar to those of their larger counterparts. Agencies in all size categories shared similar views on 11 of the 18 reforms. For example, the largest percentages of agencies in all size categories viewed the annual plan requirement as helpful to them in managing and operating their agencies. But the percentage of small agencies sharing this view was smaller than the percentages of medium and large agencies. For the remaining seven QHWRA reforms, small, medium, and large agencies' views varied. For example, small and medium agencies viewed the requirement that each housing agency have a resident member on the board of directors as neither a help nor a hindrance to them in managing and operating their agencies, while large agencies viewed the requirement as a help. Also, agencies of all sizes had similar views on factors that account for additional time spent working on HUD programs since the implementation of QHWRA. For example, the largest percentage of agencies in all size categories reported spending more time on HUD- subsidized programs after QHWRA than before the reforms were enacted, in part because of increased reporting requirements and difficulties in submitting data to HUD. Also, the largest percentage of agencies in all size categories reported contracting out about the same amount of property management and services as before QHWRA was implemented. HUD has recently issued rules to simplify and streamline regulatory requirement for small agencies. About 75 percent of small agencies reported that they believe HUD's efforts will help their operations. HUD measurement systems show mixed results for small and larger housing agencies. Fiscal year 2002 data from the Public Housing Assessment System (PHAS), HUD's system for measuring the performance of public housing agencies that manage low-rent units, show that small agencies performed slightly better than their larger counterparts. But scores for the same period from the Section Eight Management Assessment Program (SEMAP), which measures how well agencies administer Housing Choice Voucher units, indicate that small agencies did not perform as well as larger agencies. Small agencies may have lower SEMAP scores both because of the method that HUD uses to calculate scores and because small agencies generally have less experience administering these units. Fiscal year 2002 scores from HUD's risk assessment system- -the Public and Indian Housing Information Center (PIC)--show that a larger proportion of small agencies were designated as low risk than larger agencies. According to HUD officials, small agencies are more likely to be considered low risk because these agencies typically have a less complicated funding structure than larger agencies and operate less complex programs. According to HUD officials, public housing agencies receive technical assistance based on HUD's determination of their individual needs and agencies' requests for specific types of assistance. HUD uses its performance measurement and risk assessment systems--PHAS, SEMAP, and PIC--to determine housing agencies' need for technical assistance. These systems indicate that small agencies may need more assistance than their larger counterparts in administering Housing Choice Voucher units. HUD field office officials told us that the needs of small housing agencies sometimes differ from those of larger agencies. They stated, for example, that small agencies are more likely to require assistance with the day-to-day management of HUD programs because small agencies often have few staff who specialize in finance, accounting, management information systems, or other areas that are important to managing these programs. Currently, HUD does not maintain centralized, detailed information on the types of assistance housing agencies require or request from them. According to HUD, the agency is developing a tracking system that will allow it to collect such information in the future. The Acting Director of HUD's Office of Policy, Program, and Legislative Initiatives provided us with technical comments on our report, which we have incorporated as appropriate. With Some Exceptions, Small Agencies' Views of QHWRA Reforms Were Similar to Those of Larger Agencies: Our national survey of directors of public housing agencies showed that respondents from agencies in all size categories shared similar views on the effects of many QHWRA reforms on their ability to administer HUD programs and rental units.[Footnote 12] For example, we asked whether individual reforms had helped or hindered directors' ability to operate and manage their agencies. In response, directors of small, medium, and large housing agencies cited many of the same reforms as having helped them, although for some reforms the proportions of those agreeing differed by size category. Further, a large proportion of agency directors in each size category said that their staffs spent more time on HUD-subsidized programs as a result of the QHWRA reforms. Regardless of size, agencies in all size categories had similar views on the extent to which 13 factors had contributed to this increase in administrative time. These factors relate to new reporting requirements, difficulty with HUD's data systems, lack of clear guidance from HUD, and lack of resources for hiring and training staff. Small agencies noted that proposed regulatory relief would relieve some of their burden. Agency Directors in All Size Categories Saw Many of the Same QHWRA Reforms as Helping or Neither Helping Nor Hindering: Regardless of their agencies' size, the largest proportion of directors tended to agree on the effects of 11 of the 18 QHWRA provisions we included in our survey. We asked whether the 18 reforms had helped, hindered, or neither helped nor hindered the agencies in operating and managing their HUD programs. Overall, 7 reforms were generally seen as helping agencies to operate and manage their HUD programs, and 4 reforms were seen as neither helping nor hindering. The Most Frequent Responses for All Agencies Viewed Seven Reforms as Helping: As shown in figure 1, agencies indicated that 7 of the 18 reforms had helped them. Two reforms were viewed as helpful by large percentages of agencies in all size categories. Fungibility of capital and operating funds, which allows agencies to use up to 20 percent of their capital funds for operating purposes, was seen as helping almost 70 percent of small, 82 percent of medium, and 84 percent of large agencies, respectively. The repeal of federal preferences, which gives agencies more flexibility to decide who will be admitted to their public housing units, was viewed as helpful by 44, 63, and 69 percent, respectively, of small, medium, and large agencies. Figure 1: Survey Estimates on Effects of Seven QHWRA Reforms That Primarily Helped Agencies to Operate and Manage HUD Programs: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±5.8 percentage points or better. [End of figure] The popularity of these two reforms--the fungibility of capital and operating funds and the repeal of federal preferences--may be directly related to the funding shortfalls that have historically affected public housing agencies. As we noted in our 1998 report on HUD's Performance Funding System,[Footnote 13] the operating subsidies that HUD has traditionally provided to public housing agencies have not covered all operating expenses. However, the reforms appear to have helped greater proportions of medium and especially large agencies, which tend to manage more programs and to have more funding sources and a larger pool of potential tenants. More than a third of agencies may see QHWRA's flat rent reform, which allows housing agencies to charge rents based on the market value of the unit, as an important opportunity to increase their revenues and counter some shortfalls. Approximately 40 percent of agencies saw the flat rent reform as helpful. This reform may appeal more to small agencies with limited revenue sources--which may explain the slightly higher percentage of small agencies reporting this reform as helpful. Similarly, as shown in figure 1, the largest proportion of agencies of all sizes--about a third or more--saw the 5-year and annual plan reforms as helpful. However, the larger the agency, the greater the percentage that reported being helped, and nearly a third of respondents from small agencies saw these plans as a hindrance. This response is consistent with comments we received from HUD field staff we interviewed, who generally agreed that housing agencies needed some type of planning process but pointed out that small agencies did not have staff to undertake the level of effort involved in developing annual and 5-year plans. Small agencies were much less likely to view the Housing Choice Voucher program merger reform as helpful than were large agencies (45 percent compared with 82 percent, respectively.) These responses are consistent with other information from our survey, which indicated that approximately 58 percent of small agencies do not have Housing Choice Voucher units, while an estimated 90 percent of medium and large agencies do. The Most Frequent Responses for All Agencies Viewed Four Reforms as Neither Helping Nor Hindering: Based on the most frequent responses in each size category, the largest proportions of housing agencies in all size categories viewed the deconcentration, income targeting, site-based waiting list, and rent burden limits for the voucher program reforms as generally neither helping nor hindering their ability to manage and operate their agencies (fig. 2). Figure 2: Survey Estimates on Four QHWRA Reforms That Primarily Neither Helped Nor Hindered Agencies to Operate and Manage HUD Programs: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±5.9 percentage points or less. [End of figure] Of these four reforms, the predominate percentage of agencies in each size category viewed deconcentration and income targeting as neither a help nor a hindrance. However, a smaller percentage of small agencies viewed deconcentration as neither a help nor a hindrance than medium and large agencies. Also, about 29 percent of medium and large agencies view income targeting as a hindrance, whereas only 13 percent of small agencies see it as a hindrance. Agencies in Each Size Category Varied Somewhat in Their Views on Seven QHWRA Reforms: As shown in figure 3, for the remaining seven QHWRA reforms that we asked about--the pet policy, community service requirement, income disregard, resident surveys, resident advisory board, resident member on agency board, and minimum rents--views varied among agencies of different sizes. For example, a larger percentage of medium and large agencies viewed the pet policy rule as a hindrance, while the largest percentage of small agencies viewed it as neither a help nor a hindrance. Figure 3: Survey Estimates on Seven Reforms with Varied Impacts on Agency Ability to Operate and Manage HUD Programs: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±5.8 percentage points or less. [End of figure] In contrast, small and medium agencies most frequently viewed the resident advisory board, the resident member of agency board, and the resident survey reforms as neither helping nor hindering, while large agencies most frequently responded that these reforms were helpful. This response is consistent with earlier GAO findings. As reported in our May 2002 report on housing agencies' experiences preparing their fiscal year 2000 plans,[Footnote 14] HUD officials told us that agencies overall often had difficulty encouraging resident participation, even when it was actively solicited. Further, the officials pointed out that smaller agencies do not have the same pool of potential residents to draw on that large agencies have, making it more difficult to establish resident advisory boards and find residents willing to serve as member representatives on agencies' boards. Agencies of All Sizes Reported Spending More Time on HUD-Subsidized Programs: We asked agencies to compare the amount of time they were spending on HUD-subsidized programs following the implementation of the reforms with the amount of time they devoted to these programs before the implementation of QHWRA. At least 60 percent or more of all agencies, regardless of size, believed that they were spending more or much more time on HUD programs since the implementation of QHWRA. However, as shown in figure 4, a smaller percentage of small agencies (63 percent) than of large agencies (79 percent) reported spending more time administering their HUD programs. Figure 4: Survey Estimates of Time Agencies Spend Administering HUD Programs Since QHWRA Reforms: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±4.8 percentage points or less. [End of figure] These responses may reflect the fact that medium and large agencies generally have more HUD programs and thus more complex administrative responsibilities. Based on our survey, we estimate that only 10 percent of small agencies--compared with 19 percent of medium agencies and 58 percent of large agencies--had HUD programs other than the low-rent and voucher programs. We asked agencies that reported spending more time on HUD programs following QHWRA to indicate the extent to which each of 17 factors added to their administrative time. In general, we found more similarities than differences in the responses of agencies in all size categories. The factors most frequently cited as accounting for the increased time on HUD programs were increased reporting requirements, problems submitting data to HUD, and insufficient guidance from HUD. Smaller proportions of agencies of all sizes noted that other factors added to their administrative time, though the proportions differed somewhat. New Reporting Requirements and Changes in Existing Requirements: While agencies in all size categories cited increased reporting requirements and changes in reporting requirements as adding to the time they spent administering HUD programs, the proportions differed according to the type of program (see fig. 5). The largest proportions of agencies indicated that these items had increased the time spent administering the low-rent program to some or a great extent. However, small agencies cited these factors in larger proportions than medium or large agencies. Figure 5: Survey Estimates of Extent to Which Agencies Reported Increased Reporting Requirements and Changes in Existing Requirements Contributed to Increased Time Spent on HUD Programs: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±6.6 percentage points or less. [End of figure] However, questions about time spent administering the Housing Choice Voucher program elicited the opposite pattern of responses, with over 40 percent of all large agencies but only 15-20 percent of small agencies citing the requirements as increasing administrative time to a great or very great extent. About 60 percent of small agencies indicated that these questions were not applicable to them, reflecting the fact that most small agencies operate primarily low-rent housing programs, while larger agencies tend to operate both. Most agency directors whom we interviewed while preparing our questionnaire said that the increased reporting requirements associated with QHWRA were causing them to spend more time on paperwork than on their primary mission of providing safe, decent, and affordable housing to low-income residents. One respondent noted, "It takes more man-hours for processing and compiling statistical data for reporting and tracking purposes," taking up time the agency needed to address other functions. Housing agency directors we spoke with early in our review acknowledged the need for reporting to ensure accountability, but they reiterated that the level of reporting HUD requires is burdensome to performing their core mission. Difficulty Submitting Data and Accessing HUD's Computer System: Almost half or more of respondents, regardless of size, responded that accessing HUD's computer systems and submitting data to HUD on the low- rent program accounted for increased administrative time to a great or very great extent.[Footnote 15] As shown in figure 6, higher percentages of large agencies reported having difficulty accessing HUD's computer system (59 percent) than small agencies (49 percent), but the difference in percentages was more modest on the issue of submitting data on the low-rent program. Figure 6: Survey Estimates of Extent to Which Difficulty in Submitting Data on the Low-Rent Program and Accessing HUD's Computer Systems Contributed to Increased Time Spent on HUD Programs: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±6.5 percentage points or less. [End of figure] These responses are consistent with prior GAO and other reports, which have identified weaknesses in HUD's information systems. For example, both GAO and HUD's Inspector General have cited the public housing and Housing Choice Voucher information systems as management challenges.[Footnote 16] Also, during our review of HUD and housing agency experiences in preparing the fiscal year 2000 plans required by QHWRA,[Footnote 17] over 50 percent of HUD field location respondents said that the electronic submission of plans and the conversion of plans into a readable format at HUD had a negative effect on their ability to review and approve plans. HUD officials added that some housing agencies were required to make multiple submissions and that the agencies sometimes submitted hard copies to HUD field locations as a backup to submitting the plans electronically to HUD. Late or Unclear HUD Guidance and Changes in Policies: As shown in figure 7, between 45 and 53 percent of agencies, regardless of size, responded that late and unclear guidance or changes in HUD policies and regulations added to the time they spent administering HUD programs to a great or very great extent. These responses are consistent with the results of our earlier review of HUD and its housing agency experiences in preparing the fiscal year 2000 plans the QHWRA reforms require.[Footnote 18] In our prior review, more than 70 percent of respondents said that the guidance HUD provided on the process of developing agency plans was less than adequate. At that time, HUD field office directors told us that because of late and unclear guidance, they were unable to tell housing agencies how to complete agency plans. One official commented that guidance from HUD headquarters at the beginning of the planning process had not been very good and was late in getting to field locations. Another official reported at that same time that changing rules made it difficult to know what housing agencies should do in preparing the plans and what field offices should look for in reviewing them. Figure 7: Survey Estimates of Extent to Which Late and Unclear Guidance and Changes in Regulations Contributed to Increased Time Spent on HUD Programs: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±6.5 percentage points or less. [End of figure] Lack of Resources to Hire and Train Staff: In addition, the largest percentage of agencies, regardless of size, reported that the lack of resources for hiring and training staff and the need to train staff have contributed to the increased time spent on HUD programs to some extent or a great extent (see fig. 8). Figure 8: Survey Estimates of Extent to Which Lack of Resources for Hiring and Training New Staff and the Need to Train Staff Contributed to Increased Time Spent on HUD Programs: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±6.5 percentage points or less. [End of figure] However, when we asked housing agency directors whether they were contracting out more, less, or about the same share of property management activities after the QHWRA reforms than before, the largest proportions said that either they were contracting out about the same amount or the question did not apply to them (fig. 9). Figure 9: Survey Estimates of Extent to Which Agencies Report Contracting Out for Property Management and Services Since QHWRA: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±5.6 percentage points or less. [End of figure] Small Agencies Believe Regulatory Relief Will Help: HUD has recognized the burden QHWRA reforms have placed on small agencies and in August 2002 issued a proposed rule to simplify and streamline regulatory requirements for small housing agencies that administer the low-rent and voucher assistance programs. The rule was issued in final form in June 2003, after our survey was completed. According to HUD, the rule changes--including reducing the scope of annual plans and reducing the frequency of assessing the performance of small housing agencies under PHAS and SEMAP--will alleviate some of the administrative burden these agencies face. The rule indicates that to be eligible for the annual assessment relief, agencies need to be non- troubled and operate less than 250 low-rent units to receive reduced PHAS assessments, or operate less than 250 housing choice voucher units to receive reduced SEMAP assessments. Because the regulatory changes were in proposal form when we administered our survey, we asked respondents their views on the proposed relief. About three-quarters of the small agencies responding to our survey believed the proposed regulatory relief would help their operations to a moderate or great extent (see fig. 10). We noted that the majority of medium and large agencies responded that the question was not applicable. Figure 10: Survey Estimates of Extent to Which Agencies Believe HUD's Regulatory Relief Will Help: [See PDF for image] Note: All percentage estimates in this table have 95% confidence intervals ±3.8 percentage points or less. [End of figure] Scores from HUD's Performance Measurement and Risk Assessment Systems Show That Small and Larger Agencies' Performance Varies: We analyzed fiscal year 2002 scores from HUD's performance measurement systems and found that small agencies received higher scores than large agencies for administering low-rent units but lower scores for administering Housing Choice Voucher units. However, we also found that under SEMAP, which measures performance administering Housing Choice Voucher units, small agencies may have lower scores because of the way HUD calculates the scores and because small agencies generally have less experience managing these units. Further, data for the same year show that HUD's risk assessment system, PIC, which the agency uses to help target its monitoring and assistance efforts, typically assigns lower risk ratings to small agencies than to larger agencies. HUD officials stated that smaller agencies are more likely to receive lower risk ratings in part because these agencies generally operate less complex housing programs and have less complicated funding systems. Fiscal Year 2002 PHAS Scores Show That Many Small Agencies Performed Slightly Better Than Large Agencies in Operating Low-Rent Units: PHAS assesses housing agencies' performance in managing low-rent units using four indicators: (1) the physical condition of the properties, (2) the soundness of agencies' financial condition, (3) the effectiveness of management operations, and (4) the extent to which residents are satisfied with their services and living conditions. Each of the four PHAS indicators is scored individually. Public housing agencies can receive a total of 100 points: 30 each for the physical, financial, and management indicators and 10 for the resident satisfaction indicator. In assigning performance ratings, HUD uses a variety of weights: * High performers must have an overall score of more than 90 points and scores for individual indicators of at least 60 percent of the possible points for each of the four indicators. * Standard performers must have an overall score of 60 to 90 points and scores of at least 60 percent of the points for each of the physical, financial, and management indicators. * Troubled agencies have scores of less than 60 points overall or less than 60 percent of the points for more than one of the three indicators. PHAS data for fiscal year 2002 showed that small and medium agencies slightly outperformed large agencies in managing their low-rent units- -that is, larger percentages of small and medium agencies were high performers under the PHAS scoring system (fig. 11). Figure 11: Overall PHAS Scores, by Agency Size (Fiscal Year 2002): [See PDF for image] [End of figure] To identify potential differences based on size, we examined the scores for the individual indicators. We found that the better performance of small and medium agencies was largely attributable to slightly higher scores for the physical condition of their units. For example, 62 percent of small and 56 percent of medium agencies were rated as high performers on the physical condition indicator, compared with 45 percent of large agencies. According to HUD officials, small and medium agencies tend to score slightly higher on this indicator for two reasons. First, small agencies have fewer properties with features such as elevators and complicated heating and cooling systems that are difficult to maintain. Second, small housing agencies can more often provide on-site property management and maintenance than large agencies. HUD data show only slight differences among agencies of different sizes in scores for management, financial soundness, or resident satisfaction. Fiscal Year 2002 SEMAP Scores for Operating Housing Choice Voucher Units Were Lowest for Small Agencies: SEMAP is intended to help HUD measure public housing agencies' ability to operate Housing Choice Voucher units effectively and in compliance with regulations. Like PHAS, SEMAP scores agencies as high, standard, or troubled performers. High performers must earn at least 90 percent of the 150 possible points, and standard performers must earn between 60 and 89 percent of the points. Agencies with scores of less than 60 percent of the possible points are considered troubled. Scores are based on 14 performance areas, plus a deconcentration "bonus" indicator. The 14 SEMAP indicators measure how well agencies with Housing Choice Voucher units are monitoring the processes, reporting the data, inspecting the units, determining the rents, and using allocated vouchers. The additional deconcentration bonus indicator can provide 5 of the total 150 possible points for agencies in metropolitan areas where at least half of all Housing Choice Voucher units are in low poverty areas. As figure 12 shows, small housing agencies did not score as well, on average, as larger ones. Compared with their larger counterparts, a higher proportion of small agencies were considered troubled, and fewer were high performers. Figure 12: Overall SEMAP Scores, by Agency Size (Fiscal Year 2002): [See PDF for image] [End of figure] According to HUD officials, small agencies tend to receive lower SEMAP scores in part because these agencies lack economies of scale and may have difficulty coping with the increasingly complex program rules and data systems related to the Housing Choice Voucher program. However, HUD officials also told us that small agencies' scores may be lower because HUD calculates scores for small and larger agencies differently. Small housing agencies receiving less than $300,000 in federal funds are not rated on 7 of the 14 indicators that together account for approximately 55 percent of the total SEMAP score. As a result, small agencies that receive low scores on one indicator on which they are scored may find that their scores are lowered precipitously, resulting in a troubled rating. HUD officials have stated that the current method of SEMAP scoring makes small agencies more susceptible to a troubled rating. Using HUD's fiscal year 2002 SEMAP data, we estimated that 62 percent of small agencies were not rated on at least 1 of 7 indicators. Accordingly, we compared the scores for small, medium, and large agencies for the 7 indicators that all agencies were scored for. We found that small agencies did not perform as well as medium and large agencies on these indicators (table 2). Table 2: Average SEMAP Scores for Seven Indicators for Which All Agencies Are Scored, by Agency Size (Fiscal Year 2002): Indicator: Payment standards; Possible points[A]: 5; Average score: Small: 3.6; Average score: Medium: 4.1; Average score: Large: 4.4. Indicator: Annual reexaminations; Possible points[A]: 10; Average score: Small: 4.7; Average score: Medium: 5.8; Average score: Large: 5.7. Indicator: Correct tenant rents; Possible points[A]: 5; Average score: Small: 2.1; Average score: Medium: 2.7; Average score: Large: 2.5. Indicator: Precontract HQS inspections[B]; Possible points[A]: 5; Average score: Small: 1.9; Average score: Medium: 2.5; Average score: Large: 2.7. Indicator: Annual HQS inspections[C]; Possible points[A]: 10; Average score: Small: 0; Average score: Medium: 0; Average score: Large: 0. Indicator: Lease-up; Possible points[A]: 20; Average score: Small: 5.8; Average score: Medium: 6.6; Average score: Large: 8.6. Indicator: Family self-sufficiency; Possible points[A]: 10; Average score: Small: 1.1; Average score: Medium: 3.0; Average score: Large: 3.8. Indicator: Total; Possible points[A]: 65; Average score: Small: 19.2; Average score: Medium: 24.8; Average score: Large: 27.7. Source: GAO analysis of HUD's data. [A] In some cases, not all indicators may apply to agencies. For example, not all public housing agencies participate in the Family Self Sufficiency program and thus may not be scored on that particular indicator. [B] HUD stopped scoring this indicator as of March 31, 2002. [C] Presently, HUD is not scoring this indicator. [End of table] Of these 7 indicators, the most heavily weighted is the lease-up indicator, which measures how well each agency does in meeting HUD's goal of using 95 percent of all allocated vouchers. For small public housing agencies receiving less than $300,000 in federal funds, the lease-up indicator constitutes 20 out of the possible 65 points, or about 30 percent of the score. According to housing industry groups, most public housing agencies face barriers to making full use of Housing Choice Vouchers and thus to maintaining a 95-percent leasing rate. According to industry groups, these barriers may be financial (for example, cost of transportation, credit checks, and security deposits) or may involve a lack of experience in the private rental market. According to HUD officials, small housing agencies face additional burdens in attempting to achieve a 95-percent leasing rate because of the complexities of the program's rules and regulations for monitoring, reporting, inspecting, and leasing units in the private rental market. Fiscal Year 2002 PIC Risk Assessment Scores Show That Small Agencies Were the Most Likely to Receive Low-Risk Ratings: HUD developed the PIC risk assessment system during the mid-1990s to measure housing agencies' operating risk. PIC scores are based on three factors: performance (PHAS or SEMAP scores), funding, and compliance issues. Agencies' scores on each of these factors determine their risk levels. Of 100 possible points, agencies must have fewer than 44 points to be considered low risk, while those with 45 to 64 points are considered moderate risk and those with 65 to 100 points are considered high risk. The performance factor is based on the overall PHAS or SEMAP score, whichever is lower. The funding factor measures total authorized and disbursed funds and the percentage of disbursed funds already expended. The compliance factor measures open findings and audits from independent public accountants and HUD's Inspector General. As shown by figure 13, a greater percentage of small agencies were considered to be low risk, compared to medium and large agencies. Figure 13: Overall PIC Risk Assessment Scores, by Agency Size (Fiscal Year 2002): [See PDF for image] [End of figure] According to HUD officials, more small agencies are rated as low risk because they operate less complex programs, have a less complicated financial structure, and account for a smaller percentage of HUD funding. HUD's Technical Assistance Is Based on Agency Needs, Which Sometimes Vary by Agency Size: According to HUD officials, public housing agencies receive technical assistance based on HUD's determination of their needs and agencies' requests for specific types of assistance. HUD has two ways of determining the technical assistance needs of housing agencies: its risk assessment and performance measurement systems--PIC, SEMAP, and PHAS--and direct requests from agencies for specific types of assistance. Several HUD officials told us that small housing agencies frequently need different kinds of assistance than larger agencies. For example, they stated that small agencies typically need more assistance with day-to-day management issues than large agencies because small agencies tend to have few staff that specialize in finance, management information systems, and other areas that are important to managing HUD's programs. Currently, HUD does not maintain centralized, detailed information on the types of assistance housing agencies require or the kinds of assistance they request. However, a HUD official told us that the agency is developing a system that will allow it to collect such information in the future. HUD Uses Its Risk Assessment and Performance Measurement Systems to Determine Agencies' Technical Assistance Needs: HUD uses PIC, SEMAP, and PHAS scores to target technical assistance to the agencies that, according to these systems, need it most. PIC scores allow HUD to identify housing agencies that are having serious performance, funding, and compliance problems and to devise monitoring and technical assistance strategies to address such problems. For agencies designated as high-risk based on their PIC scores, HUD generally provides on-site monitoring to gain further information on their technical assistance needs. Agencies that are at moderate risk receive remote assistance mostly by telephone and e-mail. Agencies designated as low risk receive routine assistance. HUD field offices supplement their PIC analyses by assessing agencies on 15 "qualitative" factors--to help identify specific situations, events, and conditions that are not reflected in the quantitative factor score. These additional pieces of information help indicate problems as they emerge or instances of deteriorating performance. Field office staff use their "professional judgment" in applying the quality factors. For example, in assessing agencies on the "local crime rate" (one of the factors), field office staff may examine information from the local police and compare local crime rates with those of similar communities. If the crime rate where the agency is located is higher than the rates in similar communities, the agency may be rated as having a risk factor associated with crime. According to our analysis of fiscal year 2002 HUD data on the quality factors, a larger percentage of small agencies than medium and large agencies exhibited risk factors in several areas: staff turnover and training, timeliness (for example, in submitting HUD reports) and board and management issues (such as inadequate training for board members). A higher percentage of large agencies than small or medium agencies exhibited risk factors associated with operating major new programs, problems identified in audits, unfavorable media reports, and litigation issues (such as disputes with contractors). SEMAP scores for fiscal year 2002 were lowest for small agencies and, since SEMAP measures performance managing Housing Choice Voucher units, suggest that these agencies may need more technical assistance than others in this area. As previously discussed, 14.4 percent of small agencies were considered to be troubled under SEMAP, compared with 2.4 percent of medium and 3.8 percent of large agencies. Under SEMAP, troubled agencies must develop corrective action plans that address deficiencies identified by the assessment. HUD field offices review these plans to determine the type of technical assistance that is needed to correct the deficiency. We also found that troubled small and medium agencies did not perform as well as troubled large agencies on one of SEMAP's most important indicators--the "lease-up" indicator, which reflects agencies' performance in leasing available units. Of a maximum of 20 points, small and medium troubled agencies had an average score of 2.8 points, compared with 4.1 points for troubled large agencies. Most agencies that receive a low score on this indicator are likely to need technical assistance to develop a program to improve their leasing rate. According to a HUD official, although most agencies have difficulty leasing Housing Choice Voucher units, small agencies generally have greater difficulty than their larger counterparts because they have less experience managing such units. Finally, HUD uses PHAS scores to decide how to target assistance to agencies that show weaknesses in administering their low-rent units. Agencies designated as troubled under PHAS are referred to one of HUD's two Troubled Agency Recovery Centers (TARC) for follow-up. As previously indicated, while our analysis of PHAS data showed that small agencies were somewhat more likely than larger agencies to be considered high performers, it did not show significant differences between the percentages of small, medium, and large agencies that were designated as troubled. Further, small troubled agencies did not perform as well as troubled medium or large agencies on one of PHAS' four major indicators--management assessment. Specifically, out of a maximum 30 points, small troubled agencies had an average score of 14.8 compared to 21.4 for medium and 21.7 for large troubled agencies. According to HUD officials, these scores suggest troubled small agencies may need more assistance than larger agencies with managerial tasks such as completing work orders and releasing vacant units. HUD Also Responds to Requests from Individual Housing Agencies for Specific Types of Technical Assistance: HUD field offices regularly receive telephone, e-mail, or other written requests from housing agencies asking for help in a variety of areas, such as accessing and submitting data through HUD's computer systems, completing various HUD forms, and understanding HUD regulations. HUD officials told us that the department also receives a large number of calls from housing agencies, contractors, property owners, and others requesting assistance regarding PIC and PHAS. Most of these calls are made to HUD headquarters, which operates (1) a PIC help desk, which receives calls from housing agencies and others requesting assistance on matters such as how to log into the PIC system and how to interpret PIC requirements, and (2) the REAC technical assistance center, which serves as the primary point of contact for PHAS and other matters. According to a HUD official, the PIC help desk received 6,932 calls between August 2002 and August 2003, and REAC's technical assistance center receives between 9,000 and 10,000 calls monthly. HUD currently does not maintain detailed, centralized information on the types of assistance that it has determined housing agencies require or that agencies who contact HUD field offices or headquarters request. Accordingly, neither we nor HUD were able to determine this information on a national basis or by size. However, according to a HUD official, HUD is in the process of developing a tracking system that will allow it to collect such information in the near future. According to HUD field office directors we contacted, small agencies need more assistance with routine, day-to-day management of HUD housing programs than medium and large agencies.[Footnote 19] The officials added that small agencies need this assistance largely because they lack staff with specialized knowledge of finance, accounting, and other disciplines that are important to managing HUD's housing programs.[Footnote 20] These officials noted the following: * Small agencies are more likely than their larger counterparts to need assistance with things such as occupancy regulations, lease enforcement, rent calculations, and HUD's reporting requirements. Most of the calls that field offices receive from housing agencies requesting this type of assistance come from small agencies. Larger agencies typically do not need such assistance because they generally have in-house accountants, financial management specialists, information systems specialists, and others with detailed knowledge of HUD's programs and policies. Larger agencies often operate more complex programs than smaller ones and tend to need assistance with more complicated matters, such as leveraging private funds, developing partnerships with nonprofits, converting public housing units into Housing Choice Voucher units, and using low-income housing tax credits. * Because small agencies have fewer staff, they rely more heavily than larger agencies on contractors to carry out key functions in the administration of HUD's programs, such as accounting, finance, and procurement. In comparison, larger agencies are likely to have in-house staff who specialize in these areas. Public housing agencies require a highly specialized type of accounting services that are difficult to obtain, so small agencies and their contractors frequently contact HUD for technical assistance. Ultimately, small agencies' reliance on contractors to carry out functions such as accounting, finance, and the preparation of planning documents limits the agencies' own understanding of key aspects of their operations and further contributes to requests for technical assistance from HUD. * Small agencies are more likely than larger ones to have problems related to using HUD's data systems. Unlike large agencies, most small agencies do not have in-house information systems experts and are more likely to hire consultants to assist them with HUD's information systems. One field office director we spoke with stated that many small agencies also need assistance with HUD's information systems because either HUD's systems are not compatible with the agencies' systems or the agencies do not have staff with the skills needed to operate HUD's systems. We conducted our work between July 2002 and October 2003. Our work was conducted in accordance with generally accepted government auditing standards. Agency Comments: We requested comments on a draft of this report from the Secretary of HUD or his designee. On October 22, the Acting Director of HUD's Office of Policy, Program, and Legislative Initiatives provided us with technical comments on our report, which we have incorporated as appropriate. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution of this report until 30 days from the report date. At that time, we will provide copies of this report to the interested congressional committees and Members of Congress; the Secretary of Housing and Urban Development; the Director of the Office of Management and Budget; and other interested parties. In addition, the report will be available at no charge on GAO's Web site at [Hyperlink, http://www.gao.gov] http:// www.gao.gov. If you have any questions regarding this report, please contact me or Charles E. Wilson, Jr., Assistant Director, at (202) 512-8678. Key contributors to this report are listed in appendix V. Signed by: Sincerely yours, David G. Wood: Director, Financial Markets and Community Investment: [End of section] Appendixes: Appendix I: Scope and Methodology: The objectives of this study were to (1) compare how small and larger housing agencies view the impact of recent housing reforms on their ability to administer HUD programs, (2) compare small and larger agencies' performance, as measured by HUD, in administering the low- rent housing and Housing Choice Voucher programs, and (3) describe the differences in the technical assistance that small and larger public housing agencies require. For this report, we defined size as the number of low-rent units and housing voucher units combined as follows: very large, more than 6,599 units; large, 1,250 units to 6,599 units; medium, 250 units to 1,249 units; small, 100 units to 249 units; and very small, less than 100 units. To determine differences based on size and present the results of our analyses, throughout the report we collapsed the five categories into three: small, medium, and large housing agencies. Small agencies include very small and small, and large include large and very large. To compare how small and larger housing agencies view the impact of QHWRA reforms on their ability to manage their agencies, we used a mail questionnaire to survey a nationwide statistical sample of public housing agencies. Of the more than 80 QHWRA reforms, our survey focuses on 18 reforms public housing industry associations consider to have had the most impact on agency operations. Based on our sample, we produced national estimates of the impact of QHWRA reforms as perceived by agency directors. We analyzed respondents' most frequently occurring answers to our survey questions to identify similarities and differences between small, medium, and large agencies' perceptions about the impact of QHWRA reforms on their operations. Estimates based on our sample are subject to sampling error. All percentage estimates in this report have 95 percent confidence intervals of plus or minus 7 percentage points or less, unless otherwise noted. See appendix II for a more detailed discussion of the development of this survey and the sampling frame. A copy of the survey can be found in appendix III. To compare the assessed performance of small, medium, and large public housing agencies, we analyzed fiscal year 2002 PHAS data, fiscal year 2002 SEMAP data, and fiscal year 2002 PIC data, which we obtained from HUD. We used HUD's criteria for designating housing agencies as troubled, standard, or high performers under PHAS and SEMAP, and low, medium or high risk under PIC. To understand how HUD applies its criteria for determining performance and risk levels, we reviewed HUD guidance and interviewed HUD officials. HUD collects data for PHAS, SEMAP, and PIC once a year on a rolling quarterly basis. We requested and received the most current data available from these systems. We assessed the reliability of these data by (1) reviewing existing information about the systems and the data, (2) interviewing agency officials knowledgeable about the data, and (3) performing electronic testing of the data elements used in the report. We determined that the data were reliable enough for purposes of this report. To determine differences in the technical assistance that HUD provides to small, medium, and large housing agencies, we interviewed HUD headquarters and field office officials within the Office of Public and Indian Housing. To obtain information on the technical assistance that field offices provide to housing agencies across the country, we randomly selected a limited sample of nine field office directors. We also analyzed fiscal year 2002 PHAS, SEMAP, and PIC data that HUD uses to target technical assistance to those in need. [End of section] Appendix II: Methodology for GAO's Survey of Local Housing Agencies on QHWRA Housing Reforms and Initiatives: This study's primary objective was to compare how small and larger housing agencies view the impact of recent housing reforms on their ability to administer HUD programs. To address this objective, we surveyed a stratified random sample of public housing agency directors based on size. We developed and administered a survey designed to obtain the directors' views on issues associated with QHWRA's effects on agency operations, staffing and resources, and management. We received 1,119 completed, usable surveys. The Study Population: We used end of fiscal year 2002 data from HUD's Office of Public and Indian Housing databases to identify the number of agencies nationwide[Footnote 21]. Since our primary interest was to compare responses based on size, we also requested and received the various definitions HUD uses to classify housing agencies by size. We assessed the quality of HUD's electronic data by testing for internal consistency; validating the data using other sources; and, to the extent possible, reviewing the associated documentation. Based on these tests, we determined that the data were sufficiently accurate for our purposes. The Sample Design: We used a single-stage stratified random sample of the directors of local housing agencies nationwide. For the original stratification, we used HUD's criterion, which is the number of low-rent housing units. We separated agencies into five strata: 100 low-rent units or less, 101 to 249 low-rent units, 250 to 1,250 low-rent units, 1,251 to 6,599 low- rent rent units, and 6,600 or more low-rent units. We were also able to identify a sixth stratum of agencies with an unknown number of low-rent units or with voucher units only. Our total sample included 1,611 agency directors. Of the 1,611 that were in our sample, we received a total of 1,119 valid and usable surveys, for an overall response rate of 69 percent. Table 3: Survey Sample Size and Disposition: Stratum: 100 low-rent units or less; Population size: 1,587; Sample size: 413; Respondents: 262; Response rate: 63%. Stratum: 101 to 249 low-rent units; Population size: 768; Sample size: 343; Respondents: 253; Response rate: 74%. Stratum: 250 to 1,250 low-rent units; Population size: 673; Sample size: 326; Respondents: 257; Response rate: 79%. Stratum: 1,251 to 6,599 low-rent units; Population size: 114; Sample size: 114; Respondents: 91; Response rate: 80%. Stratum: 6,600 or more low-rent units; Population size: 44; Sample size: 44; Respondents: 32; Response rate: 73%. Stratum: Other and Voucher Units; Population size: 1,028; Sample size: 371; Respondents: 224; Response rate: 60%. Stratum: Total; Population size: 4,214; Sample size: 1,611; Respondents: 1,119; Response rate: 69%. Source: GAO analysis of HUD data. [End of table] Developing the Survey: To identify specific reform issues that were areas of concern, we met with officials from the public housing industry and held group discussions with 19 directors from small, medium, and large public housing agencies throughout the country. We met with directors of housing agencies at conferences of the Public Housing Authorities Directors Association (PHADA) and the National Association of Housing and Redevelopment Officials (NAHRO). During these discussion sessions, we explored (1) the challenges directors face, (2) what is being done to meet the challenges, and (3) how PHAS and QHWRA have impacted their agencies. We used a nominal group technique to prioritize the most important challenges directors of small housing agencies face. We developed our survey, in part based on our meetings with directors and officials of the public housing agencies. Specifically, the survey asked about 18 reform issues that were identified as concerns in these group discussions. About half of the 18 concerns dealt with admissions and occupancy issues, such as tenant selection and income limits. The other half included reporting requirements and resident-related issues, such as having a resident member on the board of directors. We asked whether these QHWRA reforms had helped or hindered directors' ability to operate and manage their agencies. We also asked about staffing, management, and finances. To verify the clarity, length of time of administration, and suitability of the questions, we pretested the questionnaire with the directors of local housing agencies. We also sought feedback from NAHRO and PHADA staff. A copy of the Survey of Local Housing Agencies on Housing Reforms and Initiatives can be found in appendix III. Administering the Survey: We conducted the survey between January 2003 and July 2003, using a self-administered mail-out form. We sent a second questionnaire on March 28, 2003, to all those who did not respond to our first survey in order to encourage a higher response rate. We ended data collection on July 8, 2003. We received 1,119 completed, usable surveys, for an overall response rate of 69 percent. Two surveys were eliminated because they were returned blank or indicated that they do not currently administer low- rent housing or Housing Choice Voucher programs. We used a contractor to create a database of survey responses. All data were double keyed during the data entry process, and GAO staff verified a sample of the resulting data to ensure accuracy. Nonsampling Error and Data Quality: The practical difficulties of conducting any survey can result in nonsampling errors. For example, measurement errors can be introduced if difficulties exist in interpreting a particular question or in the sources of information available to respondents in answering a question, keying in completed questionnaires, or preparing data files for analysis. We took extensive steps to minimize such errors in developing the questionnaire, collecting the data, and editing and analyzing the information. To reduce measurement error, we conducted in-depth pretesting of the questionnaire with public housing agency directors, as well as with industry officials, to make sure questions and response categories were interpreted in a consistent manner. GAO edited all surveys for consistency before they were sent for keypunching. All questionnaire responses were double key-entered into our database (that is, the entries were 100 percent verified), and a random sample of the questionnaires was further verified for completeness and accuracy. In addition to the steps taken during the development of the survey and its administration, we performed computer analyses to identify inconsistencies and other indicators of errors. When edit checks revealed inconsistent responses or individual question elements did not add correctly to the total provided, we established parameters for either calling the respondent for clarification or treating the data as missing. In addition, all computer syntax was peer reviewed and verified by separate programmers to ensure that it was written and executed correctly. Response Rates: To maximize response rates, we sent one follow-up mailing with copies of the survey on March 28, 2003, and one or two e-mail reminders to nonrespondents with known e-mail addresses. Overall, 69 percent of the sampled agencies responded to our survey, with minimal response differentials by size (see table 3 for response rates). Estimates: Estimates produced in this report are for a target population defined as directors of public housing agencies in our study population. Since one of our primary objectives involved comparing small and larger agencies, we computed estimates for our three groups of agencies-- small, medium, and large. For the report itself, we collapsed the six strata into three, combining small and very small into the small group and large and very large into the large group. We then had three groups for analysis: (1) small agencies that administered fewer than 250 low- rent and housing choice voucher units, (2) medium agencies that administered between 250 and 1,250 units, and (3) large agencies that administered more than 1,250 units. For presentation and statistical testing, we also collapsed responses. For example, in question 16, which asked about specific reforms, we combined the "greatly helped" category into the "helped" category and "greatly hindered" into "hindered" in order to derive single response categories. In question 21, we combined "moderate extent" with "some extent" and "very great extent" with "great extent." Also, because of the number of agencies answering "not applicable," "no basis to judge," or in some cases nothing at all, we chose to use this information in our analysis. Estimates were formed by weighting the survey responses to account for effective sampling rates in each stratum. These weights reflect both the initial sampling rate and the response rate for each stratum. As with most surveys, our estimation method assumes that nonrespondents would have answered like the survey respondents. We analyzed the response data on selected questions based on the most frequently occurring responses in order to identify similarities and differences in the responses across categories. Sampling Error: The results of random samples like ours are subject to sampling errors that reflect the differences between the results obtained from the samples and the results that would have been obtained from a survey of the entire population under consideration. Because we surveyed a sample of directors of public housing agencies, our results are estimates of the characteristics of public housing agencies and thus are subject to the sampling errors associated with samples of this size and type. Measurements of sampling errors are stated at a certain level of statistical confidence. GAO used the weighted results to make estimates about the entire population of local housing agencies. Our confidence in the precision of the results from this sample is expressed in 95 percent confidence intervals. The 95 percent confidence intervals are expected to include the actual results for 95 percent of the samples of this type. We calculated confidence intervals for our study results using methods that are appropriate for a stratified probability sample. For the percentages presented in this report, we are 95 percent confident that the results we would have obtained had we studied the entire study population would have been within +7 or fewer percentage points of our results, unless otherwise noted. For example, our survey estimates that 63 percent of the small agency directors believed they and their staff were spending "more or much more time" on HUD- subsidized programs after the QHWRA reforms (question 20). The 95 percent confidence interval for this estimate would be no wider than +7 percent, or from 56 percent to 70 percent. For estimates other than percentages, 95 percent confidence intervals are presented with the estimate or otherwise noted. (A modified copy of the questionnaire, showing aggregate response percentages by size, is included as appendix III.): [End of section] Appendix III Survey Results of Housing Agencies: [See PDF for image] [End of figure] [End of section] Appendix IV: Summary of 18 Reforms Contained in GAO's Survey of Public Housing Agencies: [See PDF for image] Source: HUD Directives and QHWRA Notices. [End of table] [End of section] Appendix V: GAO Contacts and Staff Acknowledgments: GAO Contacts: David G. Wood (202) 512-8678 Charles E. Wilson, Jr. (202) 512-6891: Staff Acknowledgments: In addition to the individuals named above, the following individuals made significant contributions to this report: Johnnie Barnes, Mark Braza, Emily R. Chalmers, Rudy Chatlos, Mark Egger, Gloria Hernandez- Saunders, Jamila L. Jones, Beth Koviach, John McGrail, Marc Molino, Jobenia Odum, Roberto Piñero, Mark Ramage, Terry Richardson, and Caroline Villanueva. (541022): FOOTNOTES [1] Also known as the Public Housing Reform Act. [2] Throughout this report, we refer to the Low-Rent Public Housing Assistance program as the low-rent program. Under the low-rent program, local housing agencies own and manage buildings with rental units for low-income families. Under the Housing Choice Voucher program, local housing authorities make subsidy payments on behalf of assisted households living in privately owned rental units. [3] Estimates based on our sample are subject to sampling error. Except where noted, all percentage estimates have 95 percent confidence intervals within +/-7 percentage points. See appendix II for more information. [4] Some of QHWRA's provisions went into effect when the act was enacted on October 21, 1998, while other provisions took effect later. [5] 24 CFR, Parts 902, 903, and 985, Final Rule: Deregulation for Small Housing Authorities, June 24, 2003. [6] Throughout this report, we use the term technical assistance to encompass both technical assistance and capacity building. [7] Under the Hope VI program, HUD provides assistance to help housing agencies replace and revitalize severely distressed public housing with physical, management, and social and community service improvements. Under the Capital Fund, HUD provides grants to housing agencies for capital improvement and management activities, including modernization and development of low-rent housing. [8] These surveys are administered by the housing agencies. [9] During our review, a HUD official told us that effective October 1, 2003, HUD plans to eliminate the Memphis TARC and rename the Cleveland TARC the Recovery and Prevention Corps. The Corps will serve functions similar to those of the current TARCs. [10] U.S. Department of Housing and Urban Development, The Uses of Discretionary Authority in the Public Housing Program, July 1999. [11] U.S. Department of Housing and Urban Development, The Uses of Discretionary Authority in the Tenant Based Section 8 Program, November 2000. [12] For purposes of presenting results from the survey, we say responses are "similar" for a reform if the largest proportion of estimates from all three agency size categories was for the same response. This definition is for organizing the descriptive results of the survey for presentation purposes. We conducted tests to determine the statistical significance of responses by agency size. [13] U.S. General Accounting Office, Public Housing Subsidies: Revisions to HUD's Performance Funding System Could Improve Adequacy of Funding, GAO/RCED-98-174 (Washington, D.C.: June 1998). [14] U.S. General Accounting Office, Public Housing: HUD and Public Housing Agency Experiences with Fiscal Year 2000 Plan Requirements, GAO-02-572 (Washington, D.C.: May 2002). [15] Housing agencies are required to submit information to HUD through automated systems such as the Public Housing Assessment System, the Tenant Rental Assistance Certification System, and the Multifamily Tenant Characteristics Tracking System. [16] See U.S. General Accounting Office, Major Management Challenges and Program Risks: Department of Housing and Urban Development, GAO-03- 103 (Washington, D.C.: January 2003). [17] GAO-02-572, May 2002. [18] GAO-02-572, May 2002. [19] We interviewed a total of nine HUD field office directors located in Jacksonville, Florida; Chicago, Illinois; Boston, Massachusetts; Miami, Florida; Richmond, Virginia; Atlanta, Georgia; New Orleans, Louisiana; Little Rock, Arkansas; and Newark, New Jersey. [20] Our nationwide survey of public housing agency directors found that 48 percent of small housing agencies had full-time staff working in financial management, compared to 90 percent for medium agencies and 99 percent for large agencies; 56 percent of small housing agencies had full-time staff working in property management compared to 86 percent for medium and 96 percent for large agencies; and 13 percent of small agencies had full-time staff working in information technology compared to 31 percent for medium and 89 percent for large agencies. (For additional information, see appendix III). [21] We made a minor adjustment to this database by removing two records that were no longer PHAs at the time of the survey. This resulted in 4,214 agencies in our target population. GAO's Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.