Mortgage Financing
HUD Could Realize Additional Benefits from Its Mortgage Scorecard Gao ID: GAO-06-435 April 13, 2006Along with private mortgage providers, the Department of Housing and Urban Development's (HUD) Federal Housing Administration (FHA) has been impacted by technological advances that began in the mid-1990s and that have significantly affected the way the mortgage industry works. As a result, in 2004, FHA implemented Technology Open to Approved Lenders (TOTAL) Scorecard--an automated tool that evaluates the majority of new loans insured by FHA. However, questions have emerged about the effectiveness of TOTAL. Given these concerns, you asked GAO to evaluate the way the agency developed and uses this new tool. This report looks at (1) the reasonableness of FHA's approach to developing TOTAL and (2) the potential benefits to HUD of expanding its use of TOTAL.
Some of the choices that FHA made during the development process could limit TOTAL's effectiveness, although overall the process was reasonable. Like the private sector, FHA and its contractor used many of the same variables, as well as an accepted modeling process, to develop TOTAL. However, the data that FHA and its contractors used to develop TOTAL were 12 years old by the time FHA implemented the scorecard, and the market has changed significantly since then. Also, FHA, among other things, (1) did not develop a formal plan for updating TOTAL on a regular basis; (2) did not include all the important variables that could help explain expected loan performance; and (3) selected a type of model that limits how the scorecard can be used. Despite potential problems with TOTAL, HUD could still see added benefits from it. As a result of TOTAL, FHA lenders and borrowers have seen two new benefits--less paperwork and more consistent underwriting decisions. However, FHA could gain additional benefits if, like private lenders and mortgage insurers, it put TOTAL to other uses. These uses include relying on TOTAL to help inform general management decision making, price products based on risk, and launch new products. Adopting these scorecard uses from the private sector could potentially generate three other benefits for FHA, including the ability to react to changes in the market, more control over its financial condition, and a broader customer base. Additionally, HUD's Government National Mortgage Association, a government corporation that guarantees securities of federally insured or guaranteed mortgage loans, could use credit scores that are used by TOTAL to help improve the transparency of the secondary mortgage market.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-435, Mortgage Financing: HUD Could Realize Additional Benefits from Its Mortgage Scorecard
This is the accessible text file for GAO report number GAO-06-435
entitled 'Mortgage Financing: HUD Could Realize Additional Benefits
from Its Mortgage Scorecard' which was released on May 15, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to the Chairman, Subcommittee on Housing and Community
Opportunity, Committee on Financial Services, House of Representatives:
April 2006:
Mortgage Financing:
HUD Could Realize Additional Benefits from Its Mortgage Scorecard:
GAO-06-435:
GAO Highlights:
Highlights of GAO-06-435, a report to the Chairman, Subcommittee on
Housing and Community Opportunity, Committee on Financial Services,
House of Representatives.
Why GAO Did This Study:
Along with private mortgage providers, the Department of Housing and
Urban Development‘s (HUD) Federal Housing Administration (FHA) has been
impacted by technological advances that began in the mid-1990s and that
have significantly affected the way the mortgage industry works. As a
result, in 2004, FHA implemented Technology Open to Approved Lenders
(TOTAL) Scorecard”an automated tool that evaluates the majority of new
loans insured by FHA. However, questions have emerged about the
effectiveness of TOTAL. Given these concerns, you asked GAO to
evaluate the way the agency developed and uses this new tool. This
report looks at (1) the reasonableness of FHA‘s approach to developing
TOTAL and (2) the potential benefits to HUD of expanding its use of
TOTAL.
What GAO Found:
Some of the choices that FHA made during the development process could
limit TOTAL‘s effectiveness, although overall the process was
reasonable. Like the private sector, FHA and its contractor used many
of the same variables, as well as an accepted modeling process, to
develop TOTAL. However, the data that FHA and its contractors used to
develop TOTAL were 12 years old by the time FHA implemented the
scorecard, and the market has changed significantly since then. Also,
FHA, among other things,
* did not develop a formal plan for updating TOTAL on a regular basis;
* did not include all the important variables that could help explain
expected loan performance, and;
* selected a type of model that limits how the scorecard can be used.
Despite potential problems with TOTAL, HUD could still see added
benefits from it. As a result of TOTAL, FHA lenders and borrowers have
seen two new benefits--less paperwork and more consistent underwriting
decisions. However, FHA could gain additional benefits if, like
private lenders and mortgage insurers, it put TOTAL to other uses (see
table). These uses include relying on TOTAL to help inform general
management decision making, price products based on risk, and launch
new products. Adopting these scorecard uses from the private sector
could potentially generate three other benefits for FHA, including the
ability to react to changes in the market, more control over its
financial condition, and a broader customer base. Additionally, HUD‘s
Government National Mortgage Association, a government corporation that
guarantees securities of federally insured or guaranteed mortgage
loans, could use credit scores that are used by TOTAL to help improve
the transparency of the secondary mortgage market.
Table: FHA Could Benefit Significantly More from TOTAL:
Scorecard Benefits: Past/Present Benefits: Ability to adjust
underwriting standards;
Scorecards previously used by FHA: Check;
TOTAL scorecard: Check.
Scorecard Benefits: Past/Present Benefits: Majority of loans
automatically underwritten;
Scorecards previously used by FHA: Check;
TOTAL scorecard: Check.
Scorecard Benefits: Past/Present Benefits: Faster decisions;
Scorecards previously used by FHA: Check;
TOTAL scorecard: Check.
Scorecard Benefits: Past/Present Benefits: Objective underwriting;
Scorecards previously used by FHA: Check;
TOTAL scorecard: Check.
Scorecard Benefits: Past/Present Benefits: Less paperwork for lenders;
Scorecards previously used by FHA: N/A;
TOTAL scorecard: Check.
Scorecard Benefits: Past/Present Benefits: More consistent underwriting
decisions;
Scorecards previously used by FHA: N/A;
TOTAL scorecard: Check.
Scorecard Benefits: Potential benefits: Ability to react to changes in
the market;
Scorecards previously used by FHA: N/A;
TOTAL scorecard: Check.
Scorecard Benefits: Potential benefits: More control over financial
condition;
Scorecards previously used by FHA: N/A;
TOTAL scorecard: Check.
Scorecard Benefits: Potential benefits: Broader customer base;
Scorecards previously used by FHA: N/A;
TOTAL scorecard: Check.
Source: GAO.
[End of table]
What GAO Recommends:
To improve how HUD uses and benefits from TOTAL, GAO recommends that
the Secretary of HUD (1) develop policies for updating TOTAL, including
using updated data, testing additional variables, and exploring the
benefits of alternative modeling approaches, and (2) explore additional
uses of TOTAL. HUD did not explicitly agree or disagree with our
recommendations but indicated that it was taking some steps to update
TOTAL and explore different uses for it.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-435].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact William B. Shear, (202)
512-8678, shearw@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
FHA's Approach to Developing TOTAL Was Generally Reasonable, but Some
of Its Choices Could Limit TOTAL's Effectiveness:
HUD Could Benefit Significantly More from TOTAL:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Scope and Methodology:
Appendix II: Products That Lenders Can Underwrite with TOTAL:
Appendix III: Comments from the Department of Housing and Urban
Development:
Appendix IV: GAO Contact and Staff Acknowledgments:
Table:
Table 1: TOTAL Has Generated Added Benefits:
Figure:
Figure 1: FHA's Automated Mortgage Underwriting Process:
Abbreviations:
ARM: adjustable-rate mortgage:
CLUES: Countrywide Loan Underwriting Expert System:
ECOA: Equal Credit Opportunity Act:
FHA: Federal Housing Administration:
Ginnie Mae: Government National Mortgage Association:
HUD: U.S. Department of Housing and Urban Development:
LTV: loan-to-value ratio:
MGIC: Mortgage Guaranty Insurance Corporation:
MMI: Mutual Mortgage Insurance Fund:
TOTAL: Technology Open to Approved Lenders:
United States Government Accountability Office:
Washington, DC 20548:
April 13, 2006:
The Honorable Robert W. Ney:
Chairman:
Subcommittee on Housing and Community Opportunity:
Committee on Financial Services:
House of Representatives:
Dear Mr. Chairman:
Since its inception in 1934, the Department of Housing and Urban
Development's (HUD) Federal Housing Administration (FHA) has provided
mortgage insurance for nearly 33 million properties, often for low-
income, minority, and first-time homebuyers. Along with private
mortgage providers, FHA has been impacted by technological advances
that began in the mid-1990s and that have significantly affected the
way the mortgage industry works. Among the most important of these
innovations are the automated underwriting systems that mortgage
providers now use to process loan applications.[Footnote 1] With
automated underwriting, lenders enter information on potential
borrowers into electronic systems that contain an evaluative formula,
or algorithm, called a scorecard. The scorecard uses a variety of
variables that include the borrower's characteristics (credit score and
cash reserves, for example) and loan characteristics to calculate the
applicants' creditworthiness.[Footnote 2]
In the mid-1990s, Freddie Mac and Fannie Mae developed the first
automated underwriting systems and scorecards--Freddie Mac's Loan
Prospector and Fannie Mae's Desktop Underwriter--that could be used to
evaluate applications for FHA-insured loans and inform FHA's
underwriting standards. However, these two systems' scorecards
sometimes generated conflicting results for the same borrower. In part
because FHA did not have access to these systems' proprietary
scorecards, the agency chose to replace them with its own. In addition,
HUD wanted to modernize its processes and improve its delivery to its
business partners. Between 1998 and 2004, FHA worked with HUD's
contractor, Unicon Research Corporation, to develop and implement the
Technology Open to Approved Lenders (TOTAL) scorecard. Since 2004, FHA
and its lenders have used TOTAL to evaluate applications for FHA-
insured loans and inform underwriting standards.
Recently, questions have emerged about the effectiveness of TOTAL
Scorecard, as well as concerns that FHA has not fully explored all
possible uses of this new tool. Given these concerns, you asked us to
evaluate the way the agency developed and uses this new tool. This
report looks at (1) the reasonableness of FHA's approach to developing
TOTAL and (2) the potential benefits to HUD of expanding its use of
TOTAL.
To assess the reasonableness of FHA's approach to developing TOTAL, we
reviewed agency documents and interviewed officials from HUD and Unicon
Research Corporation to determine (1) the process used to develop
TOTAL, (2) the reliability of the analysis used to evaluate it, and (3)
the methods FHA used to establish policies on cut points (i.e., the
points of separation within a population of mortgage scores that divide
applications that are accepted from those that are not). To assess the
benefits to FHA of expanding its use of TOTAL, we reviewed existing
research on the uses and benefits of scorecards and interviewed private
sector companies, academics, and HUD officials about these issues. We
compared FHA's use of TOTAL with the private sector's use of scorecards
in order to determine whether FHA could benefit from any private sector
practices. We also examined the extent to which opportunities exist for
FHA to extend the use of TOTAL, and the data it utilizes, throughout
HUD by sharing information with other HUD offices that could benefit
from it. Appendix I contains details of our scope and methodology, and
appendix II contains information on the products that lenders can
underwrite with TOTAL. We conducted our work in Washington, D.C.,
between April 2005 and February 2006 in accordance with generally
accepted government auditing standards.
Results in Brief:
Some of the choices FHA made during the development process could
affect TOTAL's effectiveness, although overall the process was
reasonable. Like the private sector, FHA and its contractor used
variables that reflected borrower and loan characteristics to create
TOTAL, as well as an accepted modeling process to test the variables'
accuracy in predicting default. As a result, FHA and its contractors
were able to create a scorecard similar to those used by private sector
organizations. However, certain choices made while TOTAL was being
developed and implemented could limit its effectiveness. For example,
the data that FHA and its contractors used to develop TOTAL were 12
years old by the time FHA implemented the scorecard. The market has
changed significantly since 1992, in part because many borrowers have
lower credit scores and receive down payment assistance. FHA's TOTAL
does not take these market changes into account. In addition, among
other things, FHA:
* did not develop a formal plan for updating TOTAL on a regular basis;
* did not include all the important variables that could help explain
expected loan performance;
* selected a type of model that limits the uses to which the scorecard
can be put, and:
* did not base cut points on the loan data used to develop TOTAL.
HUD could see more benefits from TOTAL scorecard by expanding its use
of this tool. As a result of TOTAL, FHA lenders and borrowers have seen
two added benefits--less paperwork and more consistent underwriting
decisions. Private lenders and mortgage insurers, however, put their
scorecards to other uses, relying on them to help inform general
management decision making, price products based on risk, launch new
products, as well as regularly updating them. By increasing their use
of scorecards, these lenders and brokers not only reduce application
time and see more consistent results from underwriters but also are
able to broaden their customer base and improve their financial
performance. Adopting these "best practices" from the private sector
could generate similar kinds of benefits for FHA. Additionally, HUD's
Government National Mortgage Association (Ginnie Mae), which guarantees
the timely payment of principal and interest on securities issued by
private institutions and backed by pools of federally insured or
guaranteed mortgage loans, could use credit scores utilized by TOTAL to
improve the transparency of the secondary market for securities backed
by FHA-insured loans.
To improve how HUD uses and benefits from TOTAL, we recommend that the
Secretary of HUD develop policies and procedures for regularly updating
TOTAL and explore additional uses of TOTAL and the credit data it
utilizes. In comments on a draft of the report, HUD did not explicitly
agree or disagree with our recommendations but indicated that it was
taking some steps to update TOTAL and explore different uses for it.
Background:
Congress established FHA in 1934 under the National Housing Act (Pub.
L. No. 73-479) to broaden homeownership, protect and shore up lending
institutions, and stimulate employment in the building industry. FHA's
single-family programs insure private lenders against losses from
borrower defaults on mortgages that meet FHA criteria and that are made
primarily to low-income, minority, and first-time homebuyers of
properties with one to four housing units. In 2004, some 77.5 percent
of FHA loans went to first-time homebuyers, and 35 percent of these
loans went to minorities. FHA insures most of its single-family
mortgages under its Mutual Mortgage Insurance Fund (MMI Fund), which is
supported by borrowers' insurance premiums.
FHA insures a variety of mortgages that cover initial home purchases,
construction and rehabilitation, and refinancing. Its primary program
is Section 203(b), the agency's standard product for single-family
dwellings. As the mortgage industry has developed products such as
adjustable-rate mortgages (ARM), FHA has followed suit and now insures
ARMs on single-family properties. FHA insures a variety of refinancing
products, including mortgages designed to promote energy efficiency.
Finally, it insures specialty mortgages, such as the Hawaiian Home
Lands mortgage, which enables eligible native Hawaiians to obtain
insurance for a mortgage on a homestead lease granted by the Department
of Hawaiian Home Lands.
Despite the products it insures, the number of loans FHA insures each
year has fallen dramatically since 2000, largely because lending for
conventional mortgage products (i.e., mortgages with no federal
insurance or guarantee) has grown much more rapidly since the late
1980s than mortgages insured by government entities such as FHA and the
Department of Veterans Affairs.[Footnote 3] As conventional markets
have grown, so has the private sector's use of automated underwriting
systems, which has streamlined the application process and allowed
lenders to more quickly assess the risk of loans. FHA began approving
specific automated underwriting systems for lenders in 1996 in an
effort to streamline its manual underwriting process. When it began
delegating underwriting tasks to approved lenders in the 1980s, lenders
manually underwrote loans before submitting the loan applications and
required documentation to an FHA field office for approval. Once
automated underwriting systems for FHA lending came into use, "direct
endorsement lenders" (i.e., lenders certified by HUD to underwrite
loans and determine their eligibility for FHA mortgage insurance
without obtaining prior review) could streamline the loan application
process by bypassing some documentation requirements.[Footnote 4]
According to FHA officials, automated underwriting has allowed FHA to
reduce the amount of time needed to approve insurance for a loan from
several days to 1 day.
The key to automated underwriting is a mortgage scorecard algorithm
that attempts to objectively measure the borrower's risk of default
quickly and efficiently by examining the data that has been entered
into the system. To underwrite a loan, lenders first enter into the
electronic system data such as application information and credit
scores. A scorecard compares these data with specific underwriting
criteria (e.g., cash reserves and credit requirements) using a
mathematical formula. Because the scorecard electronically analyzes
each variable, it can quickly predict the likelihood of default.
According to FHA officials, this process not only reduces underwriting
time but also decreases the amount of documentation needed to assess
the borrower's credit risk.
Private mortgage insurers, such as United Guaranty and Mortgage
Guaranty Insurance Corporation (MGIC), were among the first to develop
mortgage scorecards in the early 1990s. Beginning in the mid-1990s,
Freddie Mac and Fannie Mae began to create their own automated
underwriting systems and scorecards to evaluate conventional loans for
purchase.[Footnote 5] More specifically, Freddie Mac implemented its
Loan Prospector automated underwriting and scorecard tool by 1996, and
Fannie Mae implemented a similar tool, Desktop Underwriter, in
1997.[Footnote 6] Experience with these scorecards prompted Freddie Mac
in 1998 and Fannie Mae in 1999 to develop versions of these scorecards
for FHA that lenders first used to automatically underwrite FHA-insured
loans. Both entities used performance data on FHA-insured loans as part
of the loan data used to create the FHA versions of their scorecards.
However, while FHA cooperated in the development of Freddie Mac's and
Fannie Mae's scorecards for FHA-insured loans, they were nonetheless
proprietary to those entities, and some important details (e.g., the
weighting of the variables) were withheld from FHA. In addition, the
two scorecards sometimes yielded contradictory results for the same
borrower. As a result, FHA decided to replace the Loan Prospector and
Desktop Underwriter scorecards and develop its own scorecard that would
provide uniform outcomes.[Footnote 7]
Between 1998 and 2004, FHA contracted with Unicon Research Corporation
to develop TOTAL.[Footnote 8] Direct endorsement lenders now use TOTAL
in conjunction with automated underwriting systems that meet FHA
standards--Loan Prospector, Desktop Underwriter, and Countrywide Loan
Underwriting Expert System (CLUES)--to determine the likelihood of
default.[Footnote 9] Although TOTAL can determine the credit risk of a
borrower, it does not reject a loan; FHA requires lenders to manually
underwrite loans that are not accepted by TOTAL to determine if the
loan should be accepted or rejected.
FHA's automated mortgage underwriting process starts at the time that
the borrower meets with and submits information to the direct
endorsement lender for loan prequalification (see fig.1). First, the
direct endorsement lender enters the application variables, such as the
applicant's loan-to-value ratio (LTV) and debt, into the automated
underwriting system.[Footnote 10] Second, the automated underwriting
system electronically "pulls" the additional credit data required to
score the loan, which includes any bankruptcy and foreclosure
information and credit scores. Third, the automated underwriting system
transmits the data to TOTAL, which evaluates the information and
recommends whether the loan should be "referred" or "accepted." A
"refer" recommendation requires that the direct endorsement lender
manually underwrite the loan.[Footnote 11] An "accept" recommendation
means that the loan does not have to be manually underwritten to
determine the borrower's creditworthiness and, accordingly, that less
documentation will be required to process it. For example, borrowers
whose loans are accepted do not have to verify their employment history
if they have already met certain conditions, such as providing
confirmation of current employment. An accepted application must go
through an additional series of credit checks, or overrides, to ensure
that it meets all of FHA's underwriting standards. If the loan does not
pass the series of additional credit checks, the application can still
be downgraded to a "refer" for manual underwriting. Once the loan is
processed through the credit checks, the automated underwriting system
then sends the decision in a feedback document that the lender uses to
continue processing the loan application.
Figure 1: FHA's Automated Mortgage Underwriting Process:
[See PDF for image]
Source: GAO and NOVA Development(images).
[End of figure]
FHA's Approach to Developing TOTAL Was Generally Reasonable, but Some
of Its Choices Could Limit TOTAL's Effectiveness:
FHA's approach to developing TOTAL was generally reasonable, but some
of the decisions made during the development process could ultimately
limit the scorecard's effectiveness. Like the private sector, FHA and
its contractor followed an accepted process, using a variety of
variables that took into account such items as credit history and
economic conditions. As a result, TOTAL is similar to private sector
scorecards. But TOTAL's effectiveness could be limited by some of the
choices that were made during the development process, including the
fact that (1) the data FHA and its contractor used were 12 years old by
the time TOTAL was implemented, (2) FHA has not developed policies and
procedures for updating TOTAL, and (3) the benchmark analysis for
determining TOTAL's predictive capability may have been inadequate.
The Process FHA and Its Contractors Used to Develop TOTAL Was Generally
Reasonable:
Scorecards are typically developed and maintained using data with
specific characteristics and an accepted modeling process. The data--
such as, variables that reflect credit histories and loan information-
-are typically several years old and are drawn from samples of
borrowers whose characteristics resemble those of the borrowers whom
the scorecard will assess. The process used in the private sector to
develop the scorecard itself typically has four components:
* identifying the variables that best predict the likelihood of default,
* choosing a scorecard model by conducting various tests,
* validating the scorecard to ensure that it is stable (i.e.,
consistently produces reasonable results), and:
* determining the appropriate cut point for separating loans that will
be accepted from those that will be referred for manual underwriting.
Once the scorecard is complete, many private sector organizations plan
for and conduct ongoing analyses and generate reports to monitor and
update their scorecards. Analyses that help in updating scorecards
include measuring changes in the population of borrowers, the quality
of the portfolio, and the scorecard's effectiveness. Organizations may
conduct these analyses on a monthly and quarterly basis, and they may
also supplement these analyses with more in-depth reviews.
In developing TOTAL, FHA's contractor Unicon followed the four-step
process. First, it identified variables using data primarily for loans
that FHA had endorsed (i.e., approved for mortgage insurance) in 1992.
In 1998, when Unicon began developing TOTAL, FHA chose to use 1992 loan
data, which would reflect the characteristics of FHA borrowers and be
"seasoned," or old enough, to provide a sufficient number of defaults
that could be attributed to a borrower's poor creditworthiness. The
1992 sample of endorsed loans included 9,867 loans that did not result
in a claim default and 4,818 that did. Unicon tested the variables'
ability to predict claim default. Unicon determined that a number of
variables, such as credit, LTV ratio, and cash reserves should be
included in TOTAL. To determine the best type of credit variable for
FHA's purposes to include in TOTAL, Unicon and its subcontractor Fair
Isaac Corporation used 1994 and 1996 credit data to test various credit
models and confirm the results. These models included those that
measured borrowers' credit using only credit scores and more complex
models that were based on individual credit characteristics rather than
on a credit score. Based on this analysis, FHA decided that the
standard FICO credit score was a reasonable credit variable to include
in the scorecard.
Second, Unicon tested various versions of statistical models suitable
for developing scorecards. These were variations on two types of
models, "logit" and "hazard." Both models predict the probability of
default based on predictive variables that are weighted according to
their statistical importance, although the hazard model can predict
default over multiple time periods. FHA officials stated that, based on
Unicon's analyses, both models' predictive capability were about equal.
FHA chose the logit model, claiming that it was easier to implement and
that its estimates were easier to interpret.
Third, Unicon tested the stability of the model by estimating it
against a sample of loans from 1992 that had not been included in the
original 1992 data. In addition, Unicon tested the model's stability
over time by checking whether the determinants of defaults occurring
within 2 years were similar for the 1992 and 1994 application years.
Both stability tests, according to documents provided by FHA, suggested
that the model did not materially change over the 2-year period. In
addition, FHA performed a benchmark analysis by comparing the
performance of TOTAL with previously used scorecards--the FHA versions
of Freddie Mac's Loan Prospector and Fannie Mae's Desktop Underwriter-
-to determine the model's precision. According to documents provided by
FHA, TOTAL slightly outperformed the other scorecards.
Finally, FHA worked with Unicon, Freddie Mac, and Fannie Mae to
determine a cut point for TOTAL that would enable the agency to quickly
accept the majority of loan applications so that lenders could focus
their manual underwriting on the marginal, potentially riskier
borrowers. This cut point was based partly on a 1996 analysis that
Freddie Mac, in consultation with FHA, conducted on the version of the
Loan Prospector scorecard developed for FHA. According to HUD
officials, it was also consistent with cut points that had previously
been used before TOTAL was implemented. The current cut point allows
the agency to accept 65 to 70 percent of the loan applications
automatically and refer the remainder.
In a 2001 report, a consulting firm--KPMG LLP--that reviewed documents
relating to the development of TOTAL concluded that FHA adequately
supported most of its development decisions. The report focused on the
data used, the type of model selected, the determination of cut points,
and FHA's benchmark analysis.
Some Development and Implementation Choices Could Limit TOTAL's
Effectiveness:
Although FHA and its contractor used a reasonable and generally
accepted practice for developing TOTAL, some of the choices made during
that process could affect FHA's ability to maximize its use of the
scorecard.
Data Not Current:
By the time TOTAL was implemented in 2004, the loans in the development
sample were 12 years old. Best practices call for scorecards to be
based on data that are representative of the current mortgage market--
specifically, relevant data that are no more than several years old.
FHA officials told us that the relationship between TOTAL's predictive
variables and FHA borrowers' tendency to default had not changed
significantly since 1992 and that they believed the data were still
useful. However, since 1992, significant changes have occurred in the
mortgage industry that have affected the characteristics of those
applying for FHA-insured loans. These changes include generally lower
credit scores, increased use of down payment assistance, and new
mortgage products that have allowed borrowers who would previously have
needed an FHA-insured loan to seek conventional mortgages. As a result,
the relationships between borrower and loan characteristics and the
likelihood of default may also have changed. For example, the
statistical relationship between the LTV ratio and the likelihood of
default may be different for borrowers who receive down payment
assistance than for those who do not.
No Plan for Regular Updates:
As noted earlier, when TOTAL was implemented in 2004, FHA officials
believed that the 1992 loan sample used to develop the scorecard still
provided an adequate basis for assessing new loan applications. The
agency's subsequent analyses of TOTAL using samples of FHA-insured
loans throughout the 1990s indicate that, for years tested, the
scorecard has performed consistently in separating loans that resulted
in insurance claims from those that did not. As a result, HUD did not
update TOTAL either before it was deployed or subsequently. However,
best practices implemented by private entities and reflected in
guidance from a bank regulator call for having formal policies to
ensure that scorecards are routinely updated. Frequent updating of
scorecards ensures that they reflect changes in consumer behaviors and
thus continue to accurately predict the likelihood of default. In
September 2004, FHA awarded another contract to Unicon to, among other
things, update TOTAL by 2007. In addition, HUD indicated that, through
its contractors, it has the capacity to update TOTAL should the need
arise and has contracts for acquiring credit data to support an update
of the scorecard. However, FHA has not developed policies and
procedures for updating TOTAL on a regular basis.
Limited Sample of Loans Used for Development and Testing:
Another potential shortcoming that could affect TOTAL's effectiveness
is the fact that FHA used only endorsed loans to develop TOTAL. Because
the data did not cover all of the possible outcomes of applying for a
loan (rejection, for example), the results could be biased. Therefore,
TOTAL will likely assess a population of applications with generally
poorer overall credit quality than the original population used to
develop the scorecard and thus may not be as effective in evaluating
applicants with poorer credit. In addition, because the sample of loans
that was used to develop TOTAL differed from the total population of
loan applications, the selection and weighting of the variables in the
scorecard could be less than optimal. For the riskier applications, the
predictive variables and associated weightings might differ from those
TOTAL currently uses. FHA officials stated that, at the time TOTAL was
being developed, they did not have another choice in the data used.
However, updating TOTAL using information on marginal loans that were
referred by the scorecard, but ultimately endorsed for FHA insurance,
could help mitigate the bias problem.
Similarly, using cut points that were based only on endorsed loans at
the time TOTAL was developed--in this case, loans that were originated
using the Loan Prospector scorecard--could mean that a higher
percentage of loans that are likely to default would be accepted rather
than referred for manual underwriting. That is, a sample of endorsed
loans does not include loans that have been rejected and thus does not
represent the total population of loans. As previously noted, the
current cut point allows FHA to accept 65 to 70 percent of the total
population of loan applications and that percentage could include
riskier loans--riskier loans that the sample did not represent because
they were referred by Loan Prospector and ultimately rejected.
Furthermore, because FHA's selection of cut points was not based on
analysis of loans accepted by TOTAL, but rather on loans accepted by
Loan Prospector, the cut points may prove to be less useful for FHA as
it attempts to manage and understand its risk. KPMG LLP--the consulting
firm that reviewed TOTAL's development in 2001--raised similar concerns.
We also found that, similar to the sample of loans used to develop
TOTAL, the sample FHA used to perform the 1996 benchmark analysis of
TOTAL consisted only of endorsed loans, rather than a broader sample
that included the riskiest loans. Partly because other loan data were
not readily available, Unicon benchmarked TOTAL against a sample of
loans originated using the Loan Prospector scorecard. This sample
consisted primarily of loans that had been accepted by the scorecard
and endorsed for FHA insurance. However, because all models perform
slightly differently (i.e., each scorecard will mistakenly accept
certain high-risk, or "bad" loans), using a prescreened sample of loans
could limit the accuracy of the benchmark analysis.[Footnote 12] The
potential effect on the benchmark analysis was to suggest that TOTAL
outperformed Loan Prospector. However, using a sample of loans that had
not been prescreened by Loan Prospector might have yielded somewhat
different results that would have more accurately represented TOTAL's
predictive capabilities.
Excluded Important Variables:
While TOTAL includes many of the variables included in other mortgage
scoring systems, it does not include a number of important variables
included in other systems. For example, the systems used by Fannie Mae
and Freddie Mac may assign higher risks to adjustable rate loans than
to fixed-rate loans. ARMs are generally considered to be higher risk
than otherwise comparable fixed-rate mortgages, because borrowers are
subject to higher payments if interest rates rise. Further, other
scoring systems often include indicators for property type (single-
family detached, two-to four-unit, or condominiums, for example). FHA
indicated that these variables were not included in TOTAL because the
risk associated with them did not differ significantly in the 1992 data
used to estimate the model. However, the 1992 data set was fairly
small--fewer than 15,000 loans--and only about 16 percent of it
consisted of ARMs.[Footnote 13] In addition, condominiums and multiunit
properties are a small component of FHA's business. The modeling effort
may have failed to find significant effects for these variables simply
because of the small numbers of loans with these characteristics in the
development sample. Previous research by FHA contractors on larger
samples of FHA loans found that ARMs from this period were riskier than
comparable fixed-rate mortgages.[Footnote 14] The fact that FHA's
scoring system does not consider the extra risk inherent in ARMs or
distinguish between different types of properties, while competitors'
systems do, could have important consequences. If marginal applications
that are ARMs or multiunit properties are rejected by competitors'
systems, but accepted by FHA's, then FHA's share of these riskier loans
may increase. Finally, FHA does not include the source of the down
payment in its scorecard.[Footnote 15] However, research by HUD
contractors, HUD's Inspector General, and us have all identified the
source of a down payment as an important indicator of risk, and the use
of down payment assistance in the FHA program has grown rapidly over
the last 5 years.[Footnote 16] For example, as we reported in November
2005, FHA-insured loans with down payment assistance have higher
delinquency and insurance claim rates than do similar loans without
such assistance.
Limited Logit Model:
FHA chose a logit rather than a hazard model as a basis for TOTAL and,
therefore, potentially limited the variety of uses to which the
scorecard can be put. While a logit model predicts the probability of
default for a specific point in time, a hazard model, as previously
noted, predicts the probability of default over multiple time periods.
Because a hazard model captures the dynamic between time and loan
performance, HUD could use it to project cash flows over time and
estimate profitability. In addition, a hazard model more readily
accepts and analyzes recent data, and FHA could update a scorecard
developed from this model with recent origination data as often as it
needs. Moreover, with a relatively current scorecard, FHA could monitor
market changes and TOTAL's effectiveness at predicting defaults in the
current climate. Despite the added capabilities of a hazard model, FHA
officials stated that the logit model was sufficient for TOTAL's
intended purpose because TOTAL was only intended to be used to rank
order applications for FHA-insured loans based on the likelihood of
default.
HUD Could Benefit Significantly More from TOTAL:
FHA uses TOTAL Scorecard in much the same way as its two earlier
scorecards--to inform underwriting standards and assess loan
applications against those standards. TOTAL has produced more
consistent underwriting results and, for some lenders, has streamlined
the approval process and reduced paperwork. Private sector
organizations use their scorecards more broadly, relying on them to
assess risk, help launch new products, and broaden their customer base,
as well as updating them regularly. FHA could realize similar types of
benefits from TOTAL to help the agency serve low-and moderate-income
borrowers while ensuring its financial soundness. In addition, the
credit data used by TOTAL could help to improve the transparency of the
secondary market for FHA-insured loans.
FHA Could Realize Additional Benefits Using TOTAL:
FHA used TOTAL to test variables and identify the most predictive ones,
which the agency then used to inform its underwriting standards.
Therefore, TOTAL enables FHA to adjust its underwriting standards, if
needed, based on analyses of current market conditions--something that
Desktop Underwriter and Loan Prospector did not readily allow because
FHA did not have direct access to them. In addition, FHA directs
lenders to use TOTAL to assess loan applications by entering
information that corresponds to certain variables.[Footnote 17]As with
the previous scorecards, the only lenders that can directly interface
with TOTAL and input loan application data into the scorecard via
automated underwriting systems are direct endorsement lenders. Direct
endorsement lenders can assess most FHA loan products with TOTAL (see
app. II).
As described in table 1, FHA's current use of TOTAL has provided
additional benefits over previous scorecards, such as less paperwork
for lenders and more consistent underwriting decisions. Loan Prospector
and Desktop Underwriter had, among other things, helped speed up the
application process and provided an opportunity to base approvals on
objectively determined variables. TOTAL continues these benefits and,
in addition, has generated two others. First, as noted earlier, the
previous scorecards did not always provide consistent underwriting
decisions--that is, at times the results of their assessments differed,
which resulted in the same loan being accepted by one scorecard and
referred by the other. As a result, certain loans had to be approved
manually, through potentially subjective decision making. TOTAL limits
the number of loans that need to be approved manually because it
provides consistent automatic underwriting decisions. Second, lenders
that use TOTAL do not have to provide as much documentation for the
accepted loans they underwrite as lenders that do not use TOTAL. For
example, these lenders do not have to obtain or submit verification of
rent, and the requirements for proof of income employment and assets
are less stringent.[Footnote 18]
Table 1: TOTAL Has Generated Added Benefits:
Scorecard benefits: Ability to adjust underwriting standards:
Scorecards previously used by FHA: Check;
TOTAL scorecard: Check.
Scorecard benefits: Majority of loans automatically underwritten:
Scorecards previously used by FHA: Check;
TOTAL scorecard: Check.
Scorecard benefits: Faster decisions:
Scorecards previously used by FHA: Check;
TOTAL scorecard: Check.
Scorecard benefits: Objective underwriting:
Scorecards previously used by FHA: Check;
TOTAL scorecard: Check.
Scorecard benefits: Less paperwork for lenders:
Scorecards previously used by FHA: N/A;
TOTAL scorecard: Check.
Scorecard benefits: More consistent underwriting decisions:
Scorecards previously used by FHA: N/A;
TOTAL scorecard: Check.
Source: GAO.
[End of table]
Private Sector Organizations Benefit from Using Scorecards in a Variety
of Ways:
As noted earlier, the key to successfully using a scorecard is ensuring
that it is updated so that it can provide accurate and useful
information. Updated scorecards can provide a number of benefits
because of the variety of potential uses. Private sector organizations
we spoke with said that their scorecards had produced the same benefits
as TOTAL, including reducing loan origination times, and enhancing
consistency and objectivity in the underwriting process. In addition,
private sector organizations use their scorecards to help inform
general management decision making, set prices based on risk, and
launch new products. To inform general management decision making,
private sector organizations compare the scorecards' actual results
with its predictions to, for example, set cut points and redirect
underwriting resources from relatively low-risk cases to more marginal
borrowers. To set risk-based prices, private sector organizations use
scorecards to rank the relative risk of borrowers and price products
according to that ranking. For instance, mortgage insurers may use FICO
scores as a basis for reducing insurance premiums for low-risk
borrowers. Finally, to help launch new products, these lenders may use
scorecards to balance risk and compensating factors. For example, a
product with a more flexible LTV could be offered to borrowers with
characteristics such as a strong credit history.
As a result of these uses, private lenders have been able to broaden
their customer base and improve their financial performance. Expanding
their product offerings based on a greater understanding of risk allows
lenders to broaden their customer base. Lenders told us that their
scorecards had allowed them to underwrite some borrowers who would have
been rejected using manual underwriting and to develop products to
better serve borrowers who were at a greater risk of default. One
official noted that the scorecard had provided a greater understanding
of the individual borrower's risk and that, as a result, borrowers who
would previously have been considered for subprime loans were now rated
at a higher level of eligibility. In addition, lenders reported being
able to reduce personnel costs because the organizations were writing
fewer loans manually. Ultimately, these lenders said that they were
able to maximize their profits because of the streamlining and cost
reductions the scorecards provided.
Implementing Private Sector Scorecard Practices Could Provide
Additional Benefits for FHA:
FHA could see additional benefits from TOTAL if it implemented some
private sector practices. By routinely monitoring and updating TOTAL,
for instance, FHA could better anticipate, understand, and react to
changes in the marketplace. FHA could also exercise more control over
its financial condition by using the scorecard to help (1) project
estimated insurance claims and adjust cut points and (2) institute its
proposal for risk-based pricing of the agency's mortgage insurance
products. FHA could also use TOTAL to aid its efforts to develop new
products for underserved borrowers.
FHA could better anticipate, understand, and react to changes in the
marketplace if, like the private sector, it routinely updated TOTAL.
Updating the scorecard as new data become available could help ensure
that changes in consumer behavior are reflected in the model, which can
be affected by changes in products and other trends. By routinely
comparing the scorecard's actual results to its predictions, FHA could
ascertain whether TOTAL was effectively predicting default risk and
make any necessary changes to the variables. In addition, FHA could use
TOTAL to more accurately determine the performance of new loans, which
HUD currently monitors on an ad hoc basis, to inform policy discussions
on the creation and revision of FHA products.
FHA could exercise more control over its financial condition,
specifically its credit subsidy costs and financial soundness, by using
the scorecard's default predictions to project estimated claims and
adjust cut points if necessary.[Footnote 19] In order to project
estimated insurance claims, FHA would need to combine the variables'
weights estimated in the scorecard development process with projections
of interest and house price appreciation rates, as is done in FHA's
actuarial studies. Based on its projections, FHA could then determine
how much risk it could or should tolerate and make adjustments, if
necessary, to the cut points and thus to the numbers and types of loans
it automatically accepted and referred for manual underwriting. For
example, if FHA raised the cut point, TOTAL would accept fewer high-
risk loans (i.e., loans more likely to result in an insurance claim),
thereby lowering FHA's claim rate. Conversely, by lowering the cut
point, TOTAL would accept more high-risk loans, and the agency would
experience a higher claim rate.
TOTAL could also aid HUD's efforts to implement risk-based pricing of
its mortgage insurance products. In its fiscal year 2007 budget
submission, HUD proposed legislation that would allow the agency to
replace its current insurance premium structure, where most borrowers
pay the same premium regardless of their default risk, to a risk-based
structure where borrowers would pay higher or lower premiums depending
on their default risk. HUD believes that risk-based pricing would allow
the agency to charge more competitive mortgage insurance premiums,
attract and retain relatively low-risk borrowers, and exercise more
control over its credit subsidy costs. HUD plans to set premiums based
on an assessment of borrowers' credit histories, LTVs, and debt-to-
income ratios. However, it has not fully explored the potential of
using TOTAL--especially a version that includes additional variables,
such as down payment assistance--which is capable of evaluating risk in
a more comprehensive way, for this purpose.
In its budget submissions for fiscal years 2006 and 2007, HUD also
proposed legislative changes that would allow FHA to develop new
mortgage insurance products for low-and moderate-income borrowers
(loans with lower down payment requirements, for example). HUD believes
that its traditional customers would be better served by these new
products than some of the high-cost, nonprime products offered in the
conventional market. To the extent that FHA develops these products, it
could use TOTAL to help identify alternatives that it previously may
have believed posed too much risk, given the expected profit, when its
lenders manually underwrote loans.
Providing Data Used by TOTAL Could Offer Additional Benefits to Ginnie
Mae:
HUD's Ginnie Mae--which guarantees the timely payment of principal and
interest on securities issued by private institutions and backed by
pools of federally insured or guaranteed mortgage loans--could benefit
from the credit data used by TOTAL. As we reported in October 2005,
Ginnie Mae has taken steps to disclose more information to investors
about the FHA-insured loans that back the securities it
guarantees.[Footnote 20] However, unlike many conventional
securitizers, Ginnie Mae does not disclose credit information--for
example, summarized credit score data--for its loan pools. Disclosing
such information is important because investors can use it to more
accurately model prepayment rates. According to a Ginnie Mae official,
prior to the implementation of TOTAL in 2004, the credit scores
associated with FHA-insured loans were not available within HUD.
Because borrowers' credit scores are used by TOTAL, Ginnie Mae has
expressed interest in obtaining this information and summarizing it for
investors.
Conclusions:
Although FHA has helped to provide financing for nearly 33 million
properties, its share of the single-family market has steadily
decreased over time. Many of these potential borrowers--typically,
first-time homebuyers with minimal cash for down payments and lower
than average credit scores--may have been lost to conventional lenders.
These lenders have been, in part, able to provide conventional
mortgages to these borrowers with the increased use of scorecards--the
evaluative component of automated underwriting systems--that have
enabled them to target the traditional FHA borrower that poses the
least amount of risk. If that is the case, the effect on FHA is that it
has started to serve more high-risk borrowers. To enhance its
understanding of risk posed by its borrowers, FHA has adopted automated
underwriting and developed its own scorecard.
FHA followed an accepted process in developing TOTAL and has already
seen significant benefits from the scorecard. Because TOTAL has the
same types of capabilities as private sector scorecards, FHA has the
option to use and benefit from TOTAL in many different ways as do
private sector organizations. Specifically, FHA could use TOTAL to help
compete in the marketplace, manage risk, and serve its mission for
borrowers. TOTAL's capabilities are important to FHA, in part, because
as it begins to insure more inherently risky loans, such as loans with
down payment assistance, it needs to understand the risks they pose to
the FHA insurance fund and manage those risks.
However, the potential benefits of TOTAL cannot be realized without
ensuring that TOTAL is regularly updated and exploring additional uses
of TOTAL. For example, by not developing and implementing policies and
procedures for routinely updating TOTAL, it may become less reliable
and, therefore, less effective at predicting defaults. In addition, as
a result of not exploring additional uses of TOTAL, FHA will not
receive all of the types of benefits seen by private sector
organizations. These additional uses include applying TOTAL to proposed
initiatives--such as risk-based pricing and the development of new
products--which may help strengthen the FHA insurance fund and reach
additional borrowers. Finally, FHA has not taken steps to share credit
scores utilized by TOTAL with Ginnie Mae, which could use the
information to help improve the transparency of the secondary mortgage
market.
Recommendations for Executive Action:
To improve how HUD uses and benefits from TOTAL, we recommend that the
Secretary of HUD take the following two actions:
* develop policies and procedures for updating TOTAL on a regular
basis, including using updated data, testing additional variables,
exploring hazard model benefits, and testing other cut points; and:
* explore additional uses of TOTAL and the credit data it utilizes,
including to help adjust cut points, implement risk-based pricing,
develop new products, and enable Ginnie Mae to disclose more
information about securities backed by FHA-insured loans.
Agency Comments and Our Evaluation:
We provided HUD with a draft of this report for review and comment. HUD
provided comments in a letter from the Assistant Secretary for Housing-
Federal Housing Commissioner (see app. III). HUD made two general
observations about the report and provided specific comments on our
recommendations. First, HUD said the report did not convey the fact
that developing TOTAL was a HUD initiative to modernize its processes
and improve its delivery to business partners. Our draft report did
discuss HUD's rationale for implementing TOTAL and the scorecards that
preceded it. It also discussed the benefits of these scorecards to FHA
lenders, including less paperwork and quicker approval of mortgage
insurance. However, in response to HUD's comments, we added language to
the report that further describes HUD's motivation for developing TOTAL.
Second, HUD said that TOTAL was working exactly as envisioned (i.e.,
segregating loans requiring limited underwriting and documentation from
those requiring a full review by an individual underwriter) and that
the draft report presented no evidence that the scorecard had failed to
perform as expected. HUD also indicated that the agency had provided us
with information and analysis based on FHA loan data from the 1990s,
showing that TOTAL performed well in separating loans that resulted in
insurance claims from those that did not. Our draft report did not
state or intend to suggest that TOTAL was not fulfilling its intended
function or was not working as well as expected. In fact, the report
pointed out that TOTAL had continued the benefits of previous
scorecards while generating others. At the same time, our draft report
identified opportunities for HUD to improve TOTAL so that it could
become a more effective tool for assessing and managing risk. For
example, HUD could improve TOTAL by updating it to reflect recent
changes in the mortgage market, such as the substantial growth in the
percentage of FHA-insured loans with down payment assistance.
HUD did not explicitly agree or disagree with our recommendation that
it should develop policies and procedures for updating TOTAL, including
using updated data, testing additional variables, exploring hazard
model benefits, and testing other cut points. HUD indicated that it was
taking steps to address some aspects of our recommendation but not
others, as follows:
* HUD said that it had a formal plan for updating TOTAL, access to
TOTAL's development and implementation contractors to accommodate
updates should the need arise, and contracts for acquiring credit data
to support an update of the scorecard. As our draft report discussed,
HUD had a contract to update TOTAL by 2007. However, best practices
implemented by private entities and reflected in guidance from a bank
regulator call for having formal policies to ensure that scorecards are
routinely updated. HUD's current plan calls for one update to be
completed by 2007 (7 years after HUD finalized the scorecard model) and
has no provision for subsequent updates. Accordingly, we continue to
believe that HUD should develop policies and procedures for updating
TOTAL on a regular basis.
* HUD acknowledged that it had used 1992 data to develop TOTAL but
stated that the data spanned a wide range of credit scores and
application factors represented in greater or lesser numbers in later
cohorts of loans. We disagree that the 1992 loan data sufficiently
represents later cohorts of loans and thus continue to believe that HUD
should use more current loan data to update TOTAL. As our draft report
stated, significant changes have occurred in the mortgage industry
since 1992 that have affected the characteristics of those applying for
FHA-insured loans. These changes include generally lower credit scores,
increased use of down payment assistance, and new mortgage products
that have allowed borrowers who would have previously needed an FHA-
insured loan to seek conventional mortgages.
* HUD said that in developing TOTAL, the agency and Unicon tested all
the available variables and included those that were empirically
important, consistent with Equal Credit Opportunity Act (ECOA)
regulations (which, among other things, set forth rules for evaluating
credit applications). HUD also said that it intends to re-analyze all
available variables, including, as our draft report suggested, the
source and amount of down payment assistance. We agree that HUD should
re-analyze all available variables and incorporate them into TOTAL,
consistent with ECOA requirements. Our draft report stated that HUD's
analysis of certain variables, such as loan and property type, may not
have found significant effects simply because of the small numbers of
loans in HUD's sample that were ARMs or were for condominiums or
multiunit properties. HUD could conduct future analyses with greater
statistical reliability if it were to use larger samples of loans, as
major private lending organizations do.
* HUD stated that because TOTAL was designed to assess the
creditworthiness of borrowers, the logit model was sufficient for that
purpose. However, HUD also acknowledged that a hazard model could be
used for the purposes enumerated in our draft report. Accordingly, we
continue to believe that HUD should explore the benefits of a hazard
model.
* HUD said that it did not rely solely on a 1992 sample of loans in
setting a cut point for TOTAL and that it worked with Unicon, Fannie
Mae, and Freddie Mac, using recent distributions of loans, to obtain a
cut point that was consistent with the ones already in use for FHA
lending. Our draft report did not state that HUD relied solely on a
1992 sample of loans. Rather, it indicated that the cut point was based
partly on a 1996 analysis that Freddie Mac performed in consultation
with FHA. However, in response to this comment, we added additional
language to the report describing how HUD determined the cut point. HUD
did not address the fundamental issue raised in our draft report--that
the limitations of its original analysis suggest that the agency should
test additional cut points. We continue to believe that HUD should test
other cut points based on analysis of loans accepted by TOTAL.
HUD did not explicitly agree with our recommendation that it should
explore additional uses of TOTAL, such as using it to help adjust cut
points, implement risk-based pricing, develop new products, and enable
Ginnie Mae to disclose more information about securities backed by FHA-
insured loans. However, the actions HUD said it plans to take are
consistent with our recommendation. Specifically,
* HUD said that while TOTAL was not intended for risk-based pricing,
the agency planned to explore how TOTAL might be used for that purpose.
* HUD stated that it planned to determine the benefits that TOTAL could
present in developing new products, if given the authority from
Congress.
* HUD said that it was exploring the legal ramifications of giving
Ginnie Mae the credit scores obtained using TOTAL. HUD also provided a
technical correction, which we addressed in our final report,
concerning how it stores these credit scores.
Finally, HUD stated that the draft report contained several errors and
that these errors had been previously pointed out in meetings with us.
Where appropriate, we made technical corrections and clarifications in
response to HUD's written comments and comments provided by a HUD
official at a March 2006 meeting to discuss our findings. However, we
found that many of these comments, rather than correcting any errors,
merely provided additional levels of detail that were unnecessary for
the purpose of this report.
As agreed with your office, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 30 days
from the date of this letter. At that time, we will send copies to the
Chairman and Ranking Member of the Senate Committee on Banking,
Housing, and Urban Affairs; the Chairman and Ranking Member of the
House Committee on Financial Services; and the Ranking Member of the
Subcommittee on Housing and Community Opportunity. We also will send
copies to the Secretary of Housing and Urban Development and other
interested parties and make copies available to others upon request. In
addition, this report will be available at no charge on the GAO Web
site at [Hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-8678 or shearw@gao.gov. Contact points for our
Office of Congressional Relations and Public Affairs may be found on
the last page of this report. Key contributors to this report are
listed in appendix IV.
Sincerely yours,
Signed By:
William B. Shear:
Director:
Financial Markets and Community Investment:
[End of section]
Appendix I: Scope and Methodology:
To assess the reasonableness of the Federal Housing Administration's
(FHA) approach to developing Technology Open to Approved Lenders
(TOTAL), we reviewed agency documents and interviewed the Department of
Housing and Urban Development (HUD) and contractor officials to
determine (1) the process and data used to develop TOTAL, including how
FHA identified and evaluated scorecard variables; (2) the reliability
of the analysis used to evaluate TOTAL's effectiveness in predicting
defaults; and (3) how FHA established policies on cut points and
overrides. In addition, we reviewed industry literature and interviewed
private sector officials from large (based on volume) lending and
private mortgage insurance organizations to determine the extent to
which FHA's development of TOTAL is consistent with private sector
practices.
To assess the benefits to FHA of expanding its use of TOTAL, we
reviewed existing research on the uses and benefits of scorecards and
interviewed private sector companies, academics, and HUD officials
about these issues. We also determined how FHA and lenders use TOTAL by
reviewing relevant agency guidance and reports and interviewing FHA
officials and private lenders. In doing this work, we looked for any
ways that FHA and lenders are using TOTAL differently than the
scorecards TOTAL replaced. We compared FHA's use of TOTAL with the
private sector's use of scorecards and determined whether FHA could
benefit from any private sector practices that it has not already
adopted. We also identified any opportunities that may exist for FHA to
share information with other HUD offices that could benefit from TOTAL.
We conducted our work in Washington, D.C., between April 2005 and
February 2006 in accordance with generally accepted government auditing
standards.
[End of section]
Appendix II Products That Lenders Can Underwrite with TOTAL:
Table:
Loan purpose:
* Purchase money mortgage;
* Construction-to-permanent mortgage;
* Regular refinance with credit qualifying;
* Cash-out refinances up to 85 percent of the appraised value;
* Streamline refinance;
* Credit qualifying assumptions.
FHA insurance products:
* Section 203(b)--Mortgage insurance for one-to four-family homes;
* Section 203(h)--Single-family mortgage insurance for disaster
victims;
* Section 234(c)--Mortgage insurance for condominium units;
* Section 203(k)--Rehabilitation mortgage insurance;
* Section 251--Insurance for adjustable-rate mortgages;
* Energy efficient mortgages; Section 247--Hawaiian Home Lands.
Types of properties covered:
* Single-family dwellings of one-to four- family living units;
* Manufactured homes meeting FHA's property requirements for Title II
mortgage insurance;
* Units in low-and high- rise condominium projects.
Types of mortgages covered:
* Fixed-rate mortgages;
* Adjustable-rate mortgages.
Source: FHA.
[End of table]
[End of section]
Appendix III: Comments from the Department of Housing and Urban
Development:
U.S. Department Of Housing And Urban Development:
Washington, DC 20410- 8000:
Assistant Secretary For Housing-Federal Housing Commissioner:
Mr. William B. Shear:
Director:
Financial Markets and Community Investments:
United States Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Shear:
Thank you for providing the Federal Housing Administration (FHA) the
opportunity to respond to the report entitled "HUD Could Realize
Additional Benefits from Its Mortgage Scorecard"(GAO-06-435).
Before addressing the recommendations for executive action, I must
point out that your report does not convey that developing the TOTAL
Mortgage Scorecard was an initiative by HUD to modernize its processes
and improve its delivery to its business partners. To our knowledge,
neither of the other Federal agencies involved in the mortgage industry
has undertaken similar efforts to develop loan-level automated risk
assessment processes. Rural Housing Services in fact sought FHA's
advice on scorecard building and has adopted TOTAL as its scoring
engine in its own automated underwriting environment.
TOTAL is working exactly as it was envisioned: it segregates those
loans where limited underwriting and documentation are required from
those needing a full review by a qualified individual underwriter. Like
any recent initiative, and this is only two years old, it takes time to
determine what changes are warranted. However, the fundamental test of
a mortgage scorecard's effectiveness is how well it performs in terms
of distinguishing future claims from non-claims and GAO presents no
evidence that the scorecard has failed to perform as expected.
Indeed, HUD provided GAO with benchmark information showing that TOTAL
performed extremely well at sorting future claims from non-claims
throughout the 1990s using nationally representative random samples of
FHA loans made in 1992, 1996, and 1997, as well as identifying
delinquencies for the large universe of 1998 and 1999 (accept and
refer) FHA loans processed through Freddie Mac's LP for FHA scorecard.
HUD also provided GAO with results from the ongoing scorecard update
analyses that confirmed the power of the original TOTAL scorecard to
separate claims from non-claim defaults when compared to re-estimated
versions of TOTAL using later nationally representative random samples
of FHA loans made from 1992 through 1999-the latest year for which HUD
has information on defaults that occur before the end of the fourth
year and subsequently claim.
FHA's responses to the individual recommendations are as follows:
GAO Recommendation #1: Develop policies and procedures for the updating
of TOTAL, including using updated data, testing additional variables,
exploring hazard model benefits, and test other cut points.
FHA Response:
* Develop Policies and Procedures: FHA does indeed have a formal plan
for updating TOTAL. FHA has had continuing access to TOTAL's developer,
Unicon, and implementation contractor, ATS, to accommodate updates
should the need arise and also has, through HUD's Office of Policy
Development and Research (PD&R), established formal contracts with
credit repositories to acquire archive credit data for building
analysis files for later origination cohorts of FHA loans that are to
be used in estimating updated models. While the procurement of the
contracts with the repositories proved to be a protracted effort, it
was completed and loan cohorts with credit data have been secured in
support of the scorecard update.
* Updated data: While data from 1992 that included four-year defaults
that ultimately went to claim within the subsequent 18 months were used
in estimating the relationship between default and borrower credit and
loan application factors, that data spanned a wide range of credit
scores and application factors represented in greater or lesser numbers
in later cohorts of loans. Benchmarking analyses using later data
outlined above confirmed the consistent performance of the TOTAL
scorecard through the years.
* Testing additional variables: Unicon and HUD did test all the
available variables and included all those that proved empirically
important for explaining default performance consistent with fair
lending and ECOA regulation B, which requires the scorecard to be
empirically derived using statistically sound procedures and does not
allow for the modification of scorecard coefficients to meet a priori
expectations. The variables that GAO maintains should have been in
TOTAL did not survive as empirically important indicators in relation
to other included variables. FHA is revisiting everything anew as
required by regulation in the process of re-estimation of TOTAL
including if the source and amount of gifts for the downpayment should
be added to the algorithm.
* Exploring hazard model benefits: HUD's selection of a logit model for
the TOTAL scorecard did not limit HUD with respect to other uses of
scoring technology. The object of TOTAL was to assess credit worthiness
of borrowers at application and the logit model was sufficient to that
purpose and easier to implement. Nothing precludes the use of a hazard
model for the other purposes enumerated in GAO's report.
* Testing other cut points: While HUD did analysis of cut points and
their fair lending implications in the context of the 1992 development
sample, it did not rely solely on the 1992 distribution of loans in
setting the cut point for TOTAL when it replaced the Freddie Mac and
Fannie Mae scorecards in 2004. HUD worked with Unicon, Fannie Mae, and
Freddie Mac using recent and current distributions of loans to obtain a
cut point score (with an implied maximum default probability)
consistent with cutpoints already aligned and in use on FHA lending in
the Fannie Mae and Freddie Mac scorecards. The cut point score does not
change with shifting distributions of FHA loans. More applications will
be referred to manual underwriting with distributional shifts toward
higher risk loans and more applications will be rated accepts with
shifts to lower risk applications.
GAO Recommendation #2: Explore additional uses of TOTAL and the data in
it, such as using it to help adjust cut points, implement risk-based
pricing, develop new products, and enable Ginnie Mae to disclose more
information about securities backed by FHA-insured loans.
FHA Response:
* Implement risk-based pricing: TOTAL was not intended for risk-based
pricing. However, FHA is exploring how it might be used for that
purpose. This could prove a lengthy exercise with an unknown outcome as
TOTAL now operates as an external scorecard component to differing
automated underwriting systems rather than as an internal component to
an single integrated system where risk-based pricing could be
considerably easier to develop and implement.
* Develop new products: If FHA is given authority by Congress to offer
an array of modem products designed to enhance homeownership
opportunities, it will certainly explore the benefits that TOTAL may
present in developing such products.
* Ginnie Mae: TOTAL is not where the universe of credit bureau scores
on FHA-insured mortgages reside (although most are originally obtained
via lenders choosing to score the mortgage). However, FHA is exploring
the legal ramifications of providing Ginnie Mae with credit bureau
scores from is system of records consistent with credit law.
Finally, I would note that the report contains several errors despite
our previous meetings, in which we provided clarification. The enclosed
appendix to this letter provides additional information to address
these errors.
In closing, I would like to reiterate that FHA will continue to examine
the performance of its scorecard, and take whatever steps are necessary
to make it a better tool for assessing risk and reducing the cost to
lenders that originate mortgages insured by FHA.
Sincerely,
Signed By:
Brian D. Montgomery:
Assistant Secretary for Housing-Federal Housing Commissioner:
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
William B. Shear (202) 512-8678:
Staff Acknowledgments:
In addition to the individual named above, Steve Westley, Assistant
Director; Triana Bash; Austin Kelly; Mamesho MacCaulay; John McGrail;
Mitch Rachlis; Rachel Seid; and Grant Turner made key contributions to
this report.
(250247):
[End of section]
FOOTNOTES
[1] Underwriting refers to a risk analysis that uses information
collected during the origination process to decide whether to approve a
loan. Different mortgage providers may have different underwriting
standards.
[2] Credit scores, which assign a numeric value to a borrowers' credit
history, have become a popular tool in assessing applications for
loans. They are often called "FICO scores" because most scores are
produced with software developed by Fair Isaac Corporation. FICO scores
generally range from 300 to 850, with higher scores indicating better
credit history. The lower the credit score, the more compensating
factors lenders might require to approve a loan, such as a higher down
payment or greater borrower reserves.
[3] See GAO, Housing Finance: Ginnie Mae Is Meeting Its Mission but
Faces Challenges in a Changing Marketplace, GAO-06-9 (Washington, D.C.:
Oct. 31, 2005).
[4] Direct endorsement lenders underwrite the large majority of FHA
loans.
[5] Fannie Mae and Freddie Mac are government-sponsored enterprises
that purchase mortgages from lenders across the country, financing
their purchases by borrowing or issuing securities backed by the
mortgages (mortgage-backed securities). Most of the mortgages they
purchase are conventional mortgages.
[6] In addition to Fannie Mae's and Freddie Mac's automated
underwriting systems and scorecards, other major lenders we spoke with,
such as Countrywide, also have tools that they use internally to score
conventional loans. These lending companies use TOTAL in conjunction
with external automated underwriting systems, such as Loan Prospector
and Desktop Underwriter, to underwrite FHA-insured loans.
[7] HUD rescinded lenders' authority to use the Loan Prospector and
Desktop Underwriter scorecards to underwrite FHA-insured loans once
TOTAL Scorecard was implemented in 2004. However, lenders can continue
to use Loan Prospector and Desktop Underwriter automated underwriting
systems in conjunction with TOTAL scorecard to underwrite loans.
[8] Fair Isaac Corporation was a subcontractor to Unicon in this
effort. Although Unicon was the primary contractor FHA used to help
develop TOTAL, FHA also contracted with other firms to assist with
TOTAL's implementation.
[9] CLUES is another automated underwriting system developed by
Countrywide that lenders can use in conjunction with TOTAL to
underwrite FHA-insured loans.
[10] LTV is the relationship between the loan amount and the value of
the property (the lower of the appraised value or sales price)
expressed as a percentage of the property's value.
[11] TOTAL may refer a loan that was initially accepted if certain
conditions are found (e.g., the loan would represent an excessive debt
burden to the borrower or the borrower has experienced bankruptcy or
foreclosure) that trigger an override of the initial decision.
[12] Each institution may define a "bad" loan uniquely. FHA defines a
bad loan as a loan resulting in an insurance claim that could be
attributed to a borrower's poor creditworthiness, rather than
subsequent general economic reverses, location-based market effects, or
other things unrelated to the individual borrower.
[13] By contrast, an official from a major lending organization said
that they used about 200,000 loans to develop their scorecard.
[14] See Technical Analysis Center, Inc., An Actuarial Review of the
Federal Housing Administration Mutual Mortgage Insurance Fund for
Fiscal Year 2004 (Fairfax, VA: Oct. 19, 2004).
[15] Although private sector scorecards do not generally include this
variable, other mortgage industry participants are generally more
restrictive than FHA--for instance, they do not allow down payment
assistance from sellers, even through nonprofit organizations.
[16] See Technical Analysis Center, Inc., An Actuarial Review of the
Federal Housing Administration Mutual Mortgage Insurance Fund for
Fiscal Year 2004 (Fairfax, VA: Oct. 19, 2004); Concentrance Consulting
Group, An Examination of Down Payment Gift Programs Administered by Non-
profit Organizations (Washington, D.C.: Mar. 1, 2005); HUD IG, Final
Report of Nationwide Audit Down Payment Assistance Programs, 2000-SE-
121-0001 (Washington, D.C.: Mar. 21, 2000); and GAO, Mortgage
Financing: Additional Action Needed to Manage Risks of FHA-Insured
Loans with Down Payment Assistance, GAO-06-24 (Washington, D.C.: Nov.
9, 2005).
[17] Lenders are required to obtain the following application
information: type of mortgage and terms of loan, property information,
borrower information, and employment information.
[18] Because TOTAL obtains credit information to automatically assess
applications for FHA-insured loans, FHA does not require as much
verification as it does for applications that are manually
underwritten.
[19] The credit subsidy cost is the net present value of the estimated
long-term cost to the federal government of extending or guaranteeing
credit (through FHA mortgage insurance), calculated over the life of
the loan and excluding administrative costs. Federal agencies are
required to estimate these costs as part of the annual budget process.
FHA's main single-family mortgage insurance program is supported by the
MMI Fund, which is financed through mortgage insurance premiums and
currently operates at a profit. Since 1990, the financial condition of
the fund has been assessed by measuring the economic value of the fund-
-its capital resources plus the net present value of future cash flows-
-and the related capital ratio--the economic value as a percent of the
fund's insurance-in-force.
[20] See GAO-06-9.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
