Social Security Administration
More Effort Needed to Assess Consistency of Disability Decisions Gao ID: GAO-04-656 July 2, 2004Each year, about 2.5 million people file claims with the Social Security Administration (SSA) for disability benefits. If the claim is denied at the initial level, the claimant may appeal to the hearings level. The hearings level has allowed more than half of all appealed claims, an allowance rate that has raised concerns about the consistency of decisions made at the two levels. To help ensure consistency, SSA began a "process unification" initiative in 1994 and recently announced a new proposal to strengthen its disability programs. This report examines (1) the status of SSA's process unification initiative, (2) SSA's assessments of possible inconsistencies in decisions between adjudication levels, and (3) whether SSA's new proposal incorporates changes to improve consistency in decisions between adjudication levels.
SSA has only partially implemented its process unification initiative. Although the agency initially made improvements in its policies and training intended to address inconsistency in decisions made at the two adjudication levels, it has not continued to actively pursue these efforts. Further, as part of this initiative, the agency implemented a review of hearings level decisions to identify ways to improve training and policies, but no new improvements were made as a result of the review. Finally, the agency began tests of two process changes intended to improve the consistency of decision making between the two adjudication levels. One test, which is ongoing, was not well designed and therefore will not provide conclusive results. The other test was abandoned because of implementation difficulties. SSA's assessments have not provided a clear understanding of the extent and causes of possible inconsistencies in decisions between adjudication levels. The two measures SSA uses to monitor inconsistency of decisions have weaknesses, such as not accounting for the many factors that can affect decision outcomes, and therefore do not provide a true picture of the changes in consistency. Furthermore, SSA has not sufficiently assessed the causes of possible inconsistency. For example, SSA conducted an analysis in 1994 that identified potential areas of inconsistency, but it did not employ more sophisticated techniques--such as multivariate analyses, followed by in-depth case studies--that would allow the agency to identify and address the key areas and leading causes of possible inconsistency. SSA has yet to repeat or expand upon this 10-year-old study. SSA's new proposal incorporates changes intended to improve consistency in decisions between levels. However, challenges may hinder its implementation. Most stakeholder groups for adjudicators and claimant representatives told us that a number of aspects of the proposal hold promise for improving consistency. These included one change, being tested as part of the process unification initiative, that requires state adjudicators to more fully develop and document their decisions, as well as several new changes, such as providing both adjudication levels with equal access to medical expertise. However, stakeholder groups also told us that insufficient resources and other obstacles might hinder the implementation of some changes. Adding to uncertainties about the proposal's overall success is its dependence on a new electronic folder system that would allow cases to be easily accessed by various adjudicators across the country. However, this technically complex project has not been fully tested.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-656, Social Security Administration: More Effort Needed to Assess Consistency of Disability Decisions
This is the accessible text file for GAO report number GAO-04-656
entitled 'Social Security Administration: More Effort Needed to Assess
Consistency of Disability Decisions' which was released on July 02,
2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Subcommittee on Social Security, Committee on
Ways and Means, House of Representatives:
United States General Accounting Office:
GAO:
July 2004:
SOCIAL SECURITY ADMINISTRATION:
More Effort Needed to Assess Consistency of Disability Decisions:
GAO-04-656:
GAO Highlights:
Highlights of GAO-04-656, a report to the Chairman, Subcommittee on
Social Security, Committee on Ways and Means, House of Representatives:
Why GAO Did This Study:
Each year, about 2.5 million people file claims with the Social
Security Administration (SSA) for disability benefits. If the claim is
denied at the initial level, the claimant may appeal to the hearings
level. The hearings level has allowed more than half of all appealed
claims, an allowance rate that has raised concerns about the
consistency of decisions made at the two levels. To help ensure
consistency, SSA began a ’process unification“ initiative in 1994 and
recently announced a new proposal to strengthen its disability
programs. This report examines (1) the status of SSA‘s process
unification initiative, (2) SSA‘s assessments of possible
inconsistencies in decisions between adjudication levels, and (3)
whether SSA‘s new proposal incorporates changes to improve consistency
in decisions between adjudication levels.
What GAO Found:
SSA has only partially implemented its process unification initiative.
Although the agency initially made improvements in its policies and
training intended to address inconsistency in decisions made at the
two adjudication levels, it has not continued to actively pursue these
efforts. Further, as part of this initiative, the agency implemented a
review of hearings level decisions to identify ways to improve
training and policies, but no new improvements were made as a result of
the review. Finally, the agency began tests of two process changes
intended to improve the consistency of decision making between the two
adjudication levels. One test, which is ongoing, was not well designed
and therefore will not provide conclusive results. The other test was
abandoned because of implementation difficulties.
SSA‘s assessments have not provided a clear understanding of the
extent and causes of possible inconsistencies in decisions between
adjudication levels. The two measures SSA uses to monitor
inconsistency of decisions have weaknesses, such as not accounting for
the many factors that can affect decision outcomes, and therefore do
not provide a true picture of the changes in consistency. Furthermore,
SSA has not sufficiently assessed the causes of possible inconsistency.
For example, SSA conducted an analysis in 1994 that identified
potential areas of inconsistency, but it did not employ more
sophisticated techniques”such as multivariate analyses, followed by in-
depth case studies”that would allow the agency to identify and address
the key areas and leading causes of possible inconsistency. SSA has yet
to repeat or expand upon this 10-year-old study.
SSA‘s new proposal incorporates changes intended to improve
consistency in decisions between levels. However, challenges may hinder
its implementation. Most stakeholder groups for adjudicators and
claimant representatives told us that a number of aspects of the
proposal hold promise for improving consistency. These included one
change, being tested as part of the process unification initiative,
that requires state adjudicators to more fully develop and document
their decisions, as well as several new changes, such as providing both
adjudication levels with equal access to medical expertise. However,
stakeholder groups also told us that insufficient resources and other
obstacles might hinder the implementation of some changes. Adding to
uncertainties about the proposal‘s overall success is its dependence
on a new electronic folder system that would allow cases to be easily
accessed by various adjudicators across the country. However, this
technically complex project has not been fully tested.
What GAO Recommends:
To build an effective strategy to address possible inconsistencies in
its decisions, we recommend that SSA quickly expand its assessment of
inconsistency by implementing several specific enhancements. In its
comments, SSA had some reservations concerning our findings,
conclusions, and recommendations but agreed to pilot one
recommendation and consider the others as it refines its new proposal.
We continue to believe that SSA should implement our recommendations
without delay to ensure the effectiveness of its new proposal.
www.gao.gov/cgi-bin/getrpt?GAO-04-656.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Robert Robertson, 202-
512-7215, robertsonr@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
SSA Has Partially Implemented Its Process Unification Initiative:
SSA Lacks a Clear Understanding of the Extent and Causes of
Inconsistency between Levels:
SSA's New Proposal Incorporates Efforts to Improve the Consistency of
Decisions, but Challenges May Impede Successful Implementation:
Conclusions:
Recommendations:
Agency Comments and Our Evaluation:
Appendix I: Excerpt of SSA's Testimony Announcing Its New Proposal to
Improve Its Disability Decision-Making Process:
Appendix II: Comments from the Social Security Administration:
GAO Comments:
Appendix III: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Staff Acknowledgments:
Tables:
Table 1: SSA's Process Unification Efforts:
Table 2: Newly Proposed Changes to the Disability Decision-Making
Process:
Figures:
Figure 1: SSA's Disability Decision-Making Process and Outcomes for
Fiscal Year 2003:
Figure 2: SSA Allowances at Initial Level versus Hearings Level, by
Proportion of Allowances:
Abbreviations:
AeDib: Accelerated Electronic Disability System:
ALJ: Administrative Law Judge:
DDS: Disability Determination Services:
DI: Disability Insurance:
OASDI: Old-Age, Survivors and Disability Insurance:
OHA: Office of Hearings and Appeals:
OQA: Office of Quality Assurance and Performance Assessment:
RO: Reviewing Official:
SSA: Social Security Administration:
SSI: Supplemental Security Income:
United States General Accounting Office:
Washington, DC 20548:
July 2, 2004:
The Honorable E. Clay Shaw, Jr.:
Chairman, Subcommittee on Social Security:
Committee on Ways and Means:
House of Representatives:
Dear Mr. Chairman:
The Social Security Administration (SSA) is the nation's largest
provider of income assistance to individuals with disabilities, paying
$91 billion in federal benefits to 11.4 million beneficiaries with a
disability and their families in 2003.[Footnote 1] Each year, about 2.5
million people file claims with SSA for disability benefits. State
agencies called Disability Determination Services (DDS) decide whether
claimants meet SSA's definition of disability by applying SSA's
decision-making criteria. If a DDS ultimately decides, after an initial
determination and then a reconsideration of this decision, that a
claimant does not meet SSA's definition for disability, the claimant
may appeal to the hearings level, where an SSA Administrative Law Judge
(ALJ) reviews the claim to decide if the claimant should be allowed
benefits. About one-third of disability claims denied at the state
level were appealed to the hearings level; of these, SSA's ALJs have
allowed over one-half, with annual allowance rates fluctuating between
58 percent and 72 percent since 1985. While it is appropriate that some
appealed claims, such as those in which a claimant's impairment has
worsened and prohibits work, be allowed benefits, representatives from
SSA, the Congress, and interest groups have long been concerned that
the high rate of claims allowed at the hearings level may indicate that
decision makers at the two levels are interpreting and applying SSA's
criteria differently. If this is the case, adjudicators at the two
levels may be making inconsistent decisions that result in similar
cases receiving dissimilar decisions.
Concerned about the possibility that adjudicators are making
inconsistent decisions, SSA embarked on a "process unification"
initiative in 1994 with the goal of ensuring that adjudicators at both
levels consistently apply SSA's policy guidance and make similar
decisions on similar cases. Partly on the basis of early studies of
potential causes of inconsistent decisions, SSA included in its process
unification initiative efforts to provide consistent guidance to all
adjudicators, clarify policy, provide training, test potential process
changes, and perform a new quality review of allowances decided by
ALJs. However, SSA continues to face challenges in its efforts to
provide consistent disability decisions. These challenges, and others
associated with modernizing its disability programs, contributed to our
decision to include federal disability programs on our list of high-
risk government programs.[Footnote 2] In September 2003, SSA's
Commissioner unveiled a new proposal that laid out the vision of a
long-term strategy for improving the disability decision-making process
and helping people with disabilities return to work. Several of the
changes in the new proposal are intended to improve the accuracy,
timeliness, and consistency of decisions, such as having the DDS
decision makers more fully develop and document their decisions,
providing for centralized quality review of all decisions, and
providing both adjudication levels equal access to medical expertise.
In response to your interest in the effectiveness of SSA's past and
future efforts to improve and assess the consistency of decisions
between levels, we evaluated these agency efforts. Specifically, we
examined (1) the status of SSA's process unification initiative, (2)
SSA's assessments of possible inconsistencies in decisions between
adjudication levels, and (3) whether SSA's new proposal incorporates
changes to improve consistency in decisions between adjudication
levels.
To assess the extent to which SSA has implemented its planned
activities under the initiative, we evaluated agency documentation
describing SSA's process unification efforts. To evaluate SSA's efforts
to assess consistency in decisions between levels, we interviewed
officials from SSA's Office of Quality Assurance and Performance
Assessment (OQA) and reviewed summary data and reports from SSA's
quality assurance and performance management systems, including
findings from SSA's Disability Hearings Quality Review, which is a
quality review of DDS adjudicators' and ALJs' decisions and the
associated case files. While we evaluated SSA's methods and approaches
for assessing the consistency of decisions, we generally did not trace
figures cited by SSA back to their original source documents.[Footnote
3] To further evaluate SSA's process unification efforts and its new
proposal, we interviewed selected officials who have firsthand
knowledge about these issues, including SSA officials, staff of the
Social Security Advisory Board, and leaders of SSA stakeholder groups
representing state and hearings office adjudicative staff and claimant
attorneys.[Footnote 4] We also reviewed recent testimony and other
documents on the consistency of SSA's decisions from the Social
Security Advisory Board and other stakeholder groups. We conducted our
work between February 2003 and March 2004 in accordance with generally
accepted government auditing standards. Our ability to evaluate SSA's
new proposal has been limited by a lack of detailed information, such
as specific information on how changes will be implemented and their
costs, because the agency is still in the process of developing and
refining its proposal.
Results in Brief:
SSA has partially implemented its process unification initiative. At
the beginning of its process unification efforts, the agency took
decisive steps by issuing clarifying guidance in a number of key policy
areas to all adjudicators. However, SSA ultimately abandoned its plans
to update older policy guidance and to provide a single policy guide
for both initial-and hearings-level adjudicators, concluding that these
efforts would not be cost-effective. Similarly, while SSA initially
provided extensive process unification training to adjudicators at both
levels, the scope of SSA's training efforts on this issue has since
diminished. Also as part of this initiative, SSA implemented a quality
review of ALJ allowances, in part to identify the need for new policies
and training to improve consistency, but no new improvements were made
as a result of findings from the review. Finally, the agency began two
tests to determine if process changes would improve the consistency of
decisions. However, the test of having DDS adjudicators more fully
document decisions--which is ongoing--will, because of design flaws, be
unable to provide conclusive information on how this change might
affect the consistency of decisions. SSA abandoned its other test of
having the initial level reevaluate appealed cases for which new
medical information was submitted prior to the hearing, because of
several major difficulties encountered during testing, such as
difficulty identifying cases to be sent back to the initial level.
SSA's assessments have not provided the agency with a clear
understanding of the extent and causes of possible inconsistencies in
decisions between adjudication levels. The two measures SSA uses to
monitor changes in the extent of inconsistency of decisions have
weaknesses, such as not accounting for the many factors that can affect
decision outcomes, and therefore do not provide a true picture of
changes in consistency. SSA has also not sufficiently assessed causes
of possible inconsistency. For example, 10 years ago SSA analyzed cases
in which reviewers representing the initial and hearings level
disagreed over the final decision and identified two prevalent areas of
disagreement: assessments of claimants' mental impairments and
assessments of claimants' ability to work. Although SSA continues to
collect information that would support this analysis, it has not
repeated this initial effort, nor has it expanded on it by employing
more sophisticated techniques--such as multivariate analyses, followed
by in-depth case studies--that would allow the agency to identify and
address the key areas and leading causes of possible inconsistency.
SSA's recent proposal to improve the disability decision-making process
incorporates efforts intended to address inconsistencies in decisions
between levels. However, challenges may hinder the implementation of
the proposal. Most stakeholder groups for SSA adjudicators and claimant
representatives told us that several aspects of the proposal hold
promise for improving consistency between adjudication levels. For
example, they thought that requiring state adjudicators to more fully
develop and document their decisions--a process change that SSA is
still testing under its process unification initiative--might improve
consistency. In addition, they said that other new ideas proposed by
SSA, such as centralizing medical expertise to give both adjudication
levels equal access to it, may also improve consistency. However,
stakeholders told us that insufficient resources and other obstacles
might hinder the implementation of some changes. For example, they were
concerned that, as has happened in the past, limited resources would
hinder state adjudicators' ability to more fully document their
decisions. Stakeholders also questioned SSA's ability to attract and
retain sufficient medical expertise. Adding to uncertainties about the
proposal is its dependence upon the successful implementation of a new
electronic folder system that would allow cases to be easily accessed
by adjudicators across the country. However, this technically complex
project has not been fully tested.
GAO is making several recommendations in this report to the
Commissioner of Social Security to improve SSA's methods for assessing
the inconsistency between DDS and ALJ decisions and thereby build a
more effective strategy to address potential inconsistencies. In
commenting on the draft of this report, SSA had several reservations
concerning the report's findings, conclusions, and recommendations.
Although SSA agreed to pilot one of our recommendations, the agency
believes our other recommendations need to be reevaluated as the design
of its Commissioner's new approach to disability decision making
matures. We continue to believe that SSA should begin implementing our
recommendations without delay so that it has the critical information
needed to build a new approach to decision making that will improve the
consistency of decisions between adjudication levels.
Background:
SSA operates the Disability Insurance (DI) and Supplemental Security
Income (SSI) programs--the two largest programs providing cash benefits
to people with disabilities. The law defines disability for both
programs as the inability to engage in any substantial gainful activity
by reason of a severe physical or mental impairment that is medically
determinable and is expected to last at least 12 months or result in
death. The programs have grown substantially, from 10.7 million
beneficiaries and $61 billion in benefits in 1995 to 11.4 million
beneficiaries and $91 billion in federal benefits to individuals with
disabilities in 2003.[Footnote 5] While disability benefits account for
only 15 percent of SSA's total benefit payments for its Old-Age,
Survivors and Disability Insurance (OASDI) programs, administering the
disability benefits accounted for 45 percent of the agency's annual
administrative expenses.[Footnote 6] The relatively high cost of
administering the DI program reflects the complex and demanding nature
of making disability decisions. SSA estimates that the cost of the
disability programs will rise substantially in the near future as the
baby boom generation reaches its disability-prone years.
The disability determination process begins at a field office, where an
SSA representative determines whether a claimant meets the programs'
non-medical eligibility criteria. Claims meeting these criteria are
forwarded to the state DDS to determine if a claimant meets the
agency's definition of disability. At the DDS, the disability examiner
takes the lead, or works as a team with the medical or psychological
consultants, to analyze a claimant's documentation, gather additional
evidence as appropriate, and approve or deny the claim. A denied
claimant may ask the DDS to reconsider its finding, at which point a
different DDS team reviews the claim. If the claim is denied again, the
claimant may appeal the determination to SSA's Office of Hearings and
Appeals (OHA), where it will be reviewed by an ALJ. The ALJ usually
conducts a hearing in which the claimant and others may testify and
present new evidence.[Footnote 7] In making the disability decision,
the ALJ uses information from the hearing and from the state DDS,
including the findings of the DDS medical consultant. A claimant whose
appeal is denied may request a review by SSA's Appeals Council and, if
denied again, may file suit in federal court. Figure 1 provides an
overview of SSA's disability decision-making process and outcomes for
2003.
Figure 1: SSA's Disability Decision-Making Process and Outcomes for
Fiscal Year 2003:
[See PDF for image]
Notes:
The data provided by SSA did not include the number or rate of claims
appealed to the next decision step. Without this information, we cannot
determine the extent to which claimants appealed or abandoned their
denied claims.
Twenty-five percent of the initial DDS determinations are subject to an
alternative process that does not include the reconsideration step.
Under certain specified circumstances, ALJs and Appeals Council judges
can dismiss a claim. For example, an ALJ can dismiss a claim if the
claimant's request for a hearing is not timely and lacks a good cause
for the delay.
The Appeals Council can remand a claim by returning it to an ALJ for
further proceedings and a new hearing decision.
Because of rounding, decisional outcomes may not equal 100 percent.
[End of figure]
SSA uses a sequential evaluation process when determining disability.
First, SSA field office representatives determine whether a claimant is
performing substantial gainful work.[Footnote 8] If not, DDS or ALJ
adjudicators will assess the severity of a claimant's medical
condition(s) to determine whether it meets or equals the criteria in
SSA's regulations (commonly referred to as the medical listings). For a
claimant whose conditions do not meet or equal the listings,
adjudicators then focus on the functional consequences of the
claimant's medically determined impairments--that is, whether the
claimant can perform work he or she has done in the past, and, if not,
whether the claimant can perform other work in the national economy.
Concerns about the rate of appeals for hearings, ALJs' allowance rates,
and the accuracy and consistency of ALJ decisions led the Congress to
direct SSA to conduct a study in 1980 to determine the extent to which
hearings decisions conformed to legal requirements and binding SSA
policy.[Footnote 9] Since the allowance rates at the hearings level
could be influenced by many factors, such as the introduction of new
evidence, the purpose of the 1980 study was to present the same
evidence on cases to different reviewers representing different
adjudication levels. In determining the extent to which decision makers
agreed on whether to allow or deny benefits, the study concluded that
different levels of decision makers had significantly different
allowance rates. Specifically, the ALJs decided to allow 64 percent of
the cases, whereas the SSA's central office quality assurance
reviewers, comprising medical consultants and disability examiners,
decided that only 13 percent of cases should be allowed. The study
identified several possible causes of the disparity, including
inconsistency in the standards and procedures, interpretation of the
standards, and weight given to the evidence. The study also found that
disability decisions are complex and necessarily involve some degree of
subjectivity by adjudicators.
To help address concerns raised by this and other studies,[Footnote 10]
SSA began its process unification efforts to ensure that both levels
more consistently interpreted and applied SSA's policy guidance. SSA's
plans for its process unification initiative were part of SSA's larger
effort to redesign its disability claims process and were modified over
time. SSA's process unification plans included six major efforts, as
described in table 1.
Table 1: SSA's Process Unification Efforts:
Process unification effort: Develop a single presentation of policy;
Planned activities: Publish all new guidance in the same wording to all
adjudicators; Revise older guidance; Provide one policy manual for
adjudicators.
Process unification effort: Create additional policy guidance;
Planned activities: Publish nine process unification rulings to clarify
policy areas contributing to inconsistent DDS and ALJ decisions that
are binding on all SSA adjudicators; Publish regulations and
instructions to clarify selected process unification rulings, including
(1) the weight to be given to the treating physician's opinion when
evaluating a claim, (2) the weight to be given to the DDS medical
consultant's opinion, and (3) the evaluation of the residual functional
capacity for claimants who are limited to performing less than a full
range of sedentary work; Publish a regulation to clarify the agency's
process for acquiescing to court decisions and an action plan to
implement the regulation.
Process unification effort: Provide training;
Planned activities: Provide ongoing training to all adjudicators to
increase their understanding of the three most complex disability
areas--assessing (1) the opinion evidence from physicians and others,
(2) the claimant's symptoms, and (3) the claimant's remaining capacity
to work (i.e., residual functional capacity).
Process unification effort: Fully document DDS decisions;
Planned activities: Test procedures for fully developing and
documenting DDS decisions to determine their impact on DDS accuracy,
allowance rates, and other aspects of the claims process.
Process unification effort: Remand cases awaiting a hearing to a DDS
when new medical evidence is received;
Planned activities: Test the impact of returning cases to a DDS that
are awaiting a hearing and have received new medical information to
determine if a reevaluation of the cases by medical consultants
residing at a DDS will help to improve the consistency of decisions
between levels, with a 1-year goal of remanding 100,000 cases.
Process unification effort: Review selected ALJ allowance decisions;
Planned activities: Conduct a joint Appeals Council and OQA review of
ALJ allowances that have not yet received a final decision (i.e., the
claimant has not been awarded benefits) to identify policy areas
leading to inconsistent decisions between levels.
Source: SSA documents and prior GAO reports.
[End of table]
In 1997, we reported on the possible reasons for the inconsistency of
decisions between the initial and hearings levels. Our report found
that differences in state DDSs' and ALJs' views on the claimants'
functional abilities was a key factor in explaining why ALJs allowed
benefits on appealed cases.[Footnote 11] We also reported that poorly
documented state DDS evaluations of the claims were of limited use to
ALJs and SSA quality reviews did not focus on identifying inconsistency
in decisions. To support SSA's process unification efforts, the report
recommended that SSA, using available systems and data collected so
far, move quickly ahead to implement its quality assurance initiative
to provide consistent feedback to DDS and ALJ adjudicators as soon as
possible. In addition, we recommended that SSA expand its effort to
return cases to a DDS for review when new evidence is introduced on
appeal. Last, we recommended that SSA set goals for measuring the
effectiveness of process unification in reducing inconsistent
decisions.
More recently, the Social Security Advisory Board issued a 2001 report
that identified many factors that could potentially affect the overall
consistency of disability decision making between adjudication
levels.[Footnote 12] Some of the factors the board suggested as
potentially affecting consistency included:
* the fact that most claims are decided based on a paper review of case
evidence without face-to-face contact with an adjudicator until a
claimant has an ALJ hearing,
* involvement of attorneys and other claimant representatives at the
ALJ hearing,
* the fact that claimants are allowed to introduce new evidence and
allegations at each stage of the appeals process,
* differences in quality assurance procedures applied to initial-and
hearings-level decisions,
* differences in the training given to ALJs and state examiners, and:
* lack of clear and unified policy guidance from SSA.
Despite SSA's process unification efforts and related studies to
improve the consistency of decisions, recent ALJ allowance rates--which
declined after process unification began, but started increasing in
1999 to reach 61 percent in fiscal year 2003--still raise questions as
to whether initial-and hearings-level decision makers are consistently
applying the agency's guidance. In addition to inconsistent application
of SSA's policy guidance, there are several other reasons why a large
number of ALJ allowances are made. For example, some ALJ allowances
should be expected because, by law, cases can remain open throughout
the hearings process, allowing new evidence to be submitted that may
not have been available to the state adjudicators. Such new evidence
could show that the claimant's condition has worsened and prohibits
work. Also, SSA's decision-making criteria require that a great deal of
professional judgment be applied. As a result, some allowances at the
hearings level could simply reflect the differing judgments of two
adjudicators reviewing a case. While a claimant's deteriorating health,
changes in the characteristics of a claim over time, and the complexity
of disability decisions may help to explain some of the ALJ allowances,
studies have not sufficiently explained why consistently over half the
cases appealed to the hearings level are allowed. Instead, studies
indicate that systemic differences in the assessment of claims at both
adjudication levels are contributing to the ALJ allowance rate. For
example, our 1997 report noted a difference in state DDSs' and ALJs'
views on the claimant's functional abilities was a key factor in
explaining why ALJs allowed cases on appeal.
Inconsistency in decisions may create several problems. High hearings
allowance rates may create the perception that the hearings level is
applying SSA's criteria less strictly than the initial level and create
an incentive for claimants to appeal to an ALJ for a more favorable
decision.[Footnote 13] If deserving claimants must appeal to the
hearings level for benefits, this situation increases the burden on
claimants, who must wait, on average, almost a year for a hearing
decision and frequently incur extra costs to pay for legal
representation.[Footnote 14] In addition, to the extent that the ALJ
allowance rates include inappropriate allowances, SSA could be
incurring unwarranted program costs. Although SSA has tried to address
these problems, its inability to resolve them has contributed to our
decision to include federal disability programs on our list of high-
risk government programs.[Footnote 15]
Renewing its effort to address long-standing and critical problems with
the disability programs, SSA's Commissioner recently announced a new
proposal to improve these programs. (See app. I for an excerpt of the
announcement that describes the newly proposed decision-making
process.) In addition to proposing demonstration projects that provide
work incentives and supports to help people with disabilities return to
work, SSA has proposed significant changes to both the process of
adjudicating disability claims and the structure and management of the
agency's quality management system to improve the timeliness, accuracy,
and consistency of the disability decision-making process. The agency
believes that several of these changes will help to improve consistency
between DDS and ALJ decisions. For example, SSA plans to provide more
centralized end-of-line quality reviews. According to SSA, the proposed
quality reviews should help to hold adjudicators more accountable for
their decisions and ensure that they consistently apply SSA's policies
as well as help the agency detect and amend those policy areas leading
to inconsistent decisions. Table 2 provides a description of SSA's
proposed changes to improve the disability decision-making process.
Table 2: Newly Proposed Changes to the Disability Decision-Making
Process:
Proposed change: Centralize medical expertise from the states into
regional offices and organize experts by medical specialty;
Purpose: To make quick decisions on initial claims for individuals who
are obviously disabled (e.g., those with aggressive cancers and end-
stage renal disease) and to provide equal access to medical expertise
for all adjudicators and more consistent medical review of claims.
Proposed change: Require DDSs to fully document and explain the basis
for their decisions;
Purpose: To hold DDS adjudicators accountable for providing higher-
quality and more consistent decisions and to have better information
to identify and correct problem areas leading to incorrect decisions.
Proposed change: Eliminate DDS reconsiderations;
Purpose: To reduce time taken to process claims and avoid having
claimants who are disabled drop out of the disability claims process.
Proposed change: Create a reviewing official position;
Purpose: To evaluate all appealed DDS decisions and prepare either (1)
an on-the- record allowance, (2) a recommended disallowance detailing
reasons for a denial, or (3) a prehearing report outlining the
evidence needed to fully support the claim.
Proposed change: Require ALJs to address the reviewing officials'
reports;
Purpose: To improve accountability and consistency by having ALJs
either (1) describe in detail the basis for rejecting the reviewing
official's recommended disallowance or (2) provide detailed information
on the evidence used to support allowances made in response to the
reviewing official's prehearing report.
Proposed change: Eliminate the claimant's ability to appeal ALJ
decisions to the Appeals Council;
Purpose: To reduce processing time for claims and use resources more
effectively.
Proposed change: Use of in-line quality control;
Purpose: To build quality into each level of the decision-making
process rather than rely too heavily on case reviews performed by end-
of-line quality reviewers or by the next adjudication level in response
to claimants appealing denied decisions.
Proposed change: Use of centralized quality control unit;
Purpose: To perform end-of-line reviews for all decisions, thereby
replacing regional reviews of DDS decisions, and to provide a more
balanced review of both DDS and ALJ decisions to ensure that all
adjudicators are consistently applying SSA's policies and to detect
and amend those policy areas leading to inconsistent decisions.
Proposed change: Create oversight panels that include two ALJs and one
Administrative Appeals Judge from the Appeals Council;
Purpose: To review and either affirm or reverse an ALJ decision
referred by the centralized quality control unit when the unit
disagrees with the ALJ decision.
Sources: SSA documents and agency interviews.
Note: Under the new proposal, when the agency's centralized quality
control unit and oversight panel review an ALJ decision, a claimant
cannot submit any new information to be considered by the agency.
[End of table]
SSA does not plan to implement its proposed changes before it has
successfully implemented its Accelerated Electronic Disability (AeDib)
system. This major initiative should allow adjudication staff in states
and throughout the agency, regardless of geographic location, to access
case information electronically through the use of an electronic
disability folder. The initiative is intended to reduce delays that
result from mailing, locating, and organizing paper folders. SSA also
expects this new system to provide critical management information for
analyzing and reducing inconsistencies in disability decisions. SSA is
implementing the new system and plans to give adjudicators time to
adjust to this change before implementing its new proposal. SSA's
implementation of the new proposal will therefore be no earlier than
October 2005. In the meantime, SSA continues to discuss the proposal
with stakeholders and plans to further refine it before implementation.
SSA Has Partially Implemented Its Process Unification Initiative:
SSA has partially implemented its process unification initiative.
Although the agency initially made improvements in its policies and
training intended to improve the consistency of decisions between
adjudication levels, it has not continued to actively pursue these
efforts. As part of the initiative, the agency also implemented a
review of ALJs' allowance decisions to identify additional ways to
improve training and policies, but no new changes were made as a result
of findings from the review. Finally, the agency also began two tests
of process changes to help improve the consistency of decisions, but
one ongoing test with design problems is not likely to lead to any
conclusive results and the other test has been abandoned.
SSA Made Early Progress Improving Policies and Training, but Has Not
Actively Pursued These Efforts:
While SSA initially made progress carrying out efforts to improve
policies and training to better ensure the consistency of decisions,
the agency has not continued to actively pursue these efforts. SSA
quickly accomplished most of its planned efforts to clarify policy
guidance. In 1996, SSA issued nine process unification rulings to
clarify policy areas it found to be contributing to inconsistent
decisions. For example, one ruling provided all adjudicators with
guidance on how to weigh and document their evaluation of the treating
physician's opinions when making a disability decision.[Footnote 16]
SSA successfully went through the regulatory process several years
later and published three new regulations to strengthen its process
unification rulings, but was unable to agree on a fourth regulation
regarding the weight to be given to the treating physician's opinion
when evaluating a claim.[Footnote 17]
SSA planned to develop a single presentation of policy guidance to
replace the different sources used by each level, but has since
abandoned full implementation of these plans in favor of a more limited
approach. DDS adjudicators currently follow a detailed set of policy
and procedural guidelines,[Footnote 18] whereas ALJs rely directly on
statutes, regulations, and rulings for guidance in making disability
decisions. To help ensure that inconsistent guidance was not
contributing to inconsistent DDS and ALJ decisions, SSA began issuing
guidance in the same wording to all adjudicators in 1996. Although SSA
had also planned to address differences in policy guidance issued
before 1996 and to eventually combine existing adjudication policy
documents into a single document, it ultimately decided not to take
these additional steps. According to SSA, further efforts to unify the
policy guidance used by both levels would be a massive undertaking and
not worth the cost because the guidance issued since 1996 had already
addressed important policy areas that were leading to inconsistent
decisions. While some stakeholder groups representing adjudicators
tended to agree with SSA's position, the Social Security Advisory Board
and other groups still believe the agency should take additional steps
to provide a unified policy guide to all adjudicators. Instead of
creating one policy manual for all adjudicators, SSA told us that it
plans to undertake a comprehensive effort to evaluate and improve its
disability policies to make them less susceptible to differing
interpretations and to ensure they are up to date. A more comprehensive
approach could address key weaknesses in SSA's disability program that
we previously highlighted in our performance and accountability series,
and thereby help to modernize federal disability programs to better
meet the needs of Americans with disabilities.
Early on, SSA also provided extensive cross-training of DDS and ALJ
adjudicators, although the scope of its efforts has since diminished.
To help all adjudicators understand how to appropriately apply process
unification rulings, SSA provided extensive and mandatory training in
1996 and 1997 to 15,000 disability adjudicators (including DDS
examiners, physicians, ALJs, and quality assurance staff). The training
was provided to adjudicators at all levels of the process in three of
the most complex disability areas--assessment of symptoms, treatment of
expert opinions, and assessment of claimants' remaining capacity to
work (i.e., residual functional capacity). While this training was
intended to be ongoing, SSA's training efforts have diminished
significantly since 1997. Stakeholder groups representing DDS
adjudicators told us that SSA's training does not sufficiently cover
process unification issues. In addition, our review of DDS and OHA
participation in video training revealed inconsistent participation in
training by adjudicators. To provide ongoing training to both
adjudication levels and other components involved in the claims
process, SSA has used interactive video technology. Almost all the
state DDS sites and about 85 percent of OHA offices have this
technology. However, in reviewing participation for two recent courses,
we found for those sites with this interactive technology only 31
percent of DDS sites and 16 percent of OHA sites logged on for a course
on the role of consultative examinations, and 18 percent of DDS sites
and 4 percent of OHA sites logged on for a monthly disability hour
training class.[Footnote 19] According to SSA, neither DDS nor OHA
adjudicators are generally required to attend courses. In line with
these findings, our recent report on the human capital challenges
facing DDSs found gaps in the key knowledge and skills of their
adjudicators in the same areas SSA had earlier identified as critical
to making consistent decisions, and we recommended that SSA work with
DDSs to close these gaps.[Footnote 20]
Despite SSA's early efforts to improve policy guidance and provide
training, stakeholder groups representing state adjudicators told us
that many states are not performing the additional development and
documentation of decisions required by the process unification rulings.
They also told us that the rulings have added significantly to the
time, complexity, and subjectivity of the decision-making process,
while insufficient resources have limited their ability to fully
implement the rulings' requirements. In addition, claimant lawsuits
against three state DDSs have alleged that DDS adjudicators were not
following SSA's rulings or other decision-making guidance. In settling
these lawsuits, SSA agreed to have these states fully develop and
document cases. However, according to DDS stakeholder groups, SSA has
not ensured that states have sufficient resources to meet ruling
requirements, which they believe may lead to inconsistency in decisions
among states. Furthermore, SSA's quality assurance process does not
help ensure compliance because reviewers of DDS decisions are not
required to identify and return to the DDSs cases that are not fully
documented in accordance with the rulings. SSA's procedures require
only that the reviewers return cases that have a deficiency that could
result in an incorrect decision.
SSA's Review of ALJ Allowances Has Not Resulted in Improvements to
Policy and Training:
As part of its initiative, the agency has also implemented a quality
review of ALJ decisions, but the review has not proved useful for
identifying any new changes to SSA's policies or training that would
help to address the inconsistency of decisions. This review--referred
to as the ALJ Pre-effectuation Review--involves a sequential review by
SSA's OQA and the Appeals Council of certain ALJ allowances that have
not yet been finalized (i.e., the claimant has not yet been awarded
benefits). In selecting allowances for review, OQA uses an error-prone
profile developed from its analysis of errors detected when reviewing
DDS allowances. SSA began testing the new review of ALJs' decisions in
1996 and implemented it as an annual review in 1998. From fiscal years
1998 through 2002, OQA reviewed 27,148 ALJ allowances and of these, OQA
found fault with about 35 percent and referred them to the Appeals
Council.[Footnote 21] The Appeals Council screens the allowances for
its own review and selects those in which the prior actions may not
have been proper, fair, or in accordance with the law or the ALJ's
decision was not supported by substantial evidence.[Footnote 22] If the
council finds fault with the ALJ's decision, it will deny the claimant
benefits or return the claim to the ALJ to have the identified problems
corrected. If the council does not find fault with the ALJ's decision,
the claimant will be awarded benefits.
In addition to identifying inappropriate ALJ allowances, SSA intended
to use the new quality review to identify areas of inconsistency
between adjudication levels and ways to improve policies and training
to address those inconsistencies. Specifically, OQA identified cases
where it found fault with the ALJ decision, but the Appeals Council,
after screening them, did not accept them for review. OQA then
forwarded these cases to a panel of staff from the various components
involved in SSA's claims process to determine whether the inconsistent
assessment of these cases by OQA and the Appeals Council indicated the
need to clarify policies, issue new policies, or provide training to
improve the consistency of decisions. However, according to a SSA
official, this review did not identify any new areas of inconsistency
that required improvements to policy and training. Weaknesses in the
design of the review may have contributed to SSA's inability to
identify new policy areas contributing to inconsistency. For example,
rather than reviewing a random sample of all ALJ decisions, this review
focused on allowances. Further, the review looked only at ALJ
allowances that were selected using a DDS error-prone profile, i.e., a
profile that is based upon cases in which quality reviewers did not
agree with the DDS adjudicators' decisions. As a result, SSA selected
and reviewed nonrandom allowance decisions with case characteristics
that the agency may have already suspected were associated with
inconsistent decisions. In 1999, the panel was disbanded because
members had other priorities needing attention. OQA told us that it
continued to perform a limited review of cases viewed differently by
OQA and the Appeals Council. More recently, OQA began an effort to
summarize the results of its review and expected to issue a report of
its findings in April 2004. As of April 2004, this report had not been
issued.
SSA Began Two Tests of Process Changes to Improve the Consistency of
Decisions, but Neither Test Was Successfully Completed:
SSA began two tests of potential changes to the process to help improve
the consistency of decisions, but neither test was successfully
completed. The changes tested were (1) more fully developing and
documenting decisions made at the initial level and (2) sending
appealed cases that involve new medical information back to the initial
level to be reevaluated.
SSA wanted to test having DDSs more fully develop and document
decisions because it believed that DDS decisions, especially denials,
are often not well documented. SSA wanted to test whether better
explanations of why benefits were denied would improve the accuracy of
DDS decisions and consistency of decisions between adjudication levels.
SSA first implemented a pilot of this change to explore alternatives
for developing and documenting decisions. Then SSA tested this change,
along with other process changes, in a larger test, called the
prototype initiative. Concurrently, SSA tested other process changes,
such as the elimination of a reconsideration step and a predecision DDS
interview with the claimant.
The prototype test had limitations for predicting the impact of
documented decisions. For example, SSA's decision to test several
changes together left the agency without clear information on what
impact fully developed decisions would have on the decision-making
process without the other process changes. SSA's test design also did
not build in an ALJ feedback mechanism to provide sufficient
information on the usefulness of more fully documented decisions. SSA
continues to test this change along with other changes and, despite
limited information on the best approach for and impact of this change,
currently plans to implement more fully documented decisions as part of
the Commissioner's new proposal to improve SSA's disability programs.
SSA also began, but ultimately abandoned, a test in which appealed
cases with new medical information submitted prior to the hearing were
to be sent back to the initial level so that the evidence could be
evaluated by medical consultants residing at the DDSs. Since medical
expertise resides in the DDS and not at the hearings level,[Footnote
23] SSA decided to test whether "remanding," or sending cases to the
DDS for evaluation, might result in a more consistent review of medical
evidence. SSA believed that this change, in turn, could help improve
the consistency of decisions because the new medical information might
be contributing to ALJ allowances. However, the change also had the
potential to increase the time claimants with remanded claims would
have to wait for final decisions because claims that were not allowed
by the DDSs had to be returned to OHA for hearings. SSA began remanding
cases in July 1997, with a 1-year goal of remanding 100,000 cases, but
after 10 months, it had remanded fewer than 9,000. In implementing this
test, SSA encountered several difficulties. For example, it had
difficulty identifying the claims to be remanded and ensuring the ALJs,
who had authority over the claims, would remand the claims to the DDSs.
The ALJs' resistance to remanding claims to the DDSs may be due in part
to concerns that remanding would not lead to many allowances by the
DDSs and would result in many claims being returned to OHA, thereby
increasing the time many claimants would have to wait for a final
decision from OHA. Realizing that the agency would not be able to reach
its remanding goal, the agency decided to discontinue this test.
SSA Lacks a Clear Understanding of the Extent and Causes of
Inconsistency between Levels:
SSA's assessments have not provided the agency with a clear
understanding of the extent and causes of possible inconsistencies in
decisions between adjudication levels. The two measures SSA uses to
monitor changes in the extent of inconsistency of decisions have
weaknesses and therefore do not provide a true picture of the changes
in consistency. In addition, SSA has not sufficiently assessed the
causes of possible inconsistency. The agency conducted an analysis in
1994 that identified some potential areas of inconsistency. However,
although SSA continues to collect information that would support this
analysis, it has not repeated this initial effort, nor has it expanded
on it by employing more sophisticated assessment techniques.
SSA Attempts to Monitor Changes in the Extent of Inconsistency, but the
Measures It Uses Provide an Incomplete Picture:
SSA has made some efforts to monitor changes in the extent of
inconsistency between the initial and hearings levels, including
tracking trends in allowance rates at different levels and conducting
special reviews of ALJ decisions. Together, according to SSA, these
measures and assessments suggest that the consistency between levels
has improved since the agency began implementing its process
unification initiative. However, because of methodological weaknesses,
these measures provide, at best, a partial picture of trends in the
consistency of decisions between adjudication levels.
SSA tracks trends in the proportion of all allowances decided at each
level to assess the consistency of decisions between levels. The agency
collects information on the number of allowances granted to claimants
at each level of the process, tracks the proportion of claims allowed
at the initial level relative to the hearings level, and looks at
trends in these proportions over a period of several years. According
to data from SSA, the proportion of overall allowances that occurred at
the initial level has increased since process unification was
implemented. As shown in figure 2, in fiscal year 1996, 72 percent of
all allowances were granted at the initial level. This proportion
increased in most subsequent years, and by fiscal year 2003, 77 percent
of all allowances were granted at the initial level. Officials from
OQA, the office responsible for reviewing, evaluating, and assessing
the integrity and quality of the administration of SSA's programs, view
the relative shift toward earlier allowances as an indicator that
consistency between adjudication levels has improved, and they believe
that process unification efforts have contributed to these results.
Figure 2: SSA Allowances at Initial Level versus Hearings Level, by
Proportion of Allowances:
[See PDF for image]
Note: Hearings level allowances include allowances made by ALJs and
judges from the Appeals Council.
[End of figure]
However, SSA's measure of tracking yearly changes in the proportion of
allowances at each level is a simplistic and inconclusive indicator of
trends in the consistency of decisions because it does not control for
the multitude of factors that can affect allowance rates at either
adjudication level in any given year and over time. For example, SSA
uses "snapshot" data in looking at the proportion of allowances granted
at each level, meaning that it looks at the number of claimants and
allowances at each level during a given year, rather than following a
1-year cohort of initial claimants through the entire process and
capturing the proportion of allowances for that cohort decided at each
level. Because SSA uses data that illustrate allowance rates at a given
moment in time, it captures a different pool of claimants in the
process at each level, and the resulting allowance rates are subject to
a different set of demographic and case characteristics. Over time, the
pool of claimants may change because of factors such as a downturn in
the economy, which can cause more people with less severe impairments
to claim benefits or appeal initial denial decisions. In addition,
snapshot data may be significantly affected by fluctuations in
productivity at either adjudication level caused by process changes
that are unrelated to process unification and that affect only one
level.
SSA has collected other data to further assess trends in the
consistency of decisions. Since 1993, the agency has conducted a
biennial case review as part of its Disability Hearing Quality Review
process.[Footnote 24] This review consists of medical consultants and
disability examiners in SSA's central office evaluating a sample of
ALJs' decisions plus supporting documentation to determine whether the
ALJ has adequately supported his or her decision. In evaluating the ALJ
decisions, these medical consultants and disability examiners use the
same standards as those used by initial-level adjudicators to
adjudicate claims, which are from the official SSA program policy and
operations guidance found in POMS. To some degree, therefore, the
medical examiners and disability reviewers serve as a proxy for
initial-level adjudicators, and their decisions are representative of
how initial-level examiners should be deciding claims.
While unpublished results from the biennial case reviews indicate an
increase in supportable ALJ allowances, such findings focus on the ALJ
level and therefore provide only a partial picture of trends in
consistency. The reviews indicated that medical consultants and
disability examiners have found that supportable ALJ allowances
increased from 36 percent in fiscal year 1993-94 to 57 percent in
fiscal year 1999-2000. OQA officials told us that this increase
suggests an improvement in consistency between adjudication levels
because it indicates that disability examiners using initial-level
standards and ALJs increasingly agree on how like cases should be
decided.[Footnote 25] However, SSA's assessment provides only a partial
picture because it does not reflect trend information on the extent to
which ALJs have found DDS decisions to be supportable, to ensure that
both levels are making more consistent decisions. Although the 1994
report of findings from the initial biennial case review included the
results of a special probe in which ALJs reviewed 165 DDS
reconsideration denial decisions, the sample was not representative,
and therefore results could not serve as a baseline for developing
trend information. In 2003, SSA began another probe, in which ALJs
reviewed 400 DDS reconsideration denial determinations, but the agency
does not plan to release its findings until summer 2004.
Although SSA has limited information on how ALJs view DDS decisions,
other information collected by the agency suggests that consistency of
decision making at the initial level might not be improving. For
example, OQA reviewers routinely assess the accuracy and supportability
of DDS decisions. A recent SSA study of these data shows that the
accuracy of DDS denial decisions--those decisions most likely to be
appealed to the hearings level--has declined by 4 percentage points
over a 1-year period.[Footnote 26] Another review of DDS decisions by
OQA reviewers also suggests a lack of improvement at the initial level.
Specifically, the extent to which quality reviewers found that DDS
reconsideration denials appealed to the hearings level were supported
declined from 71 percent in fiscal year 1993-94 to 68 percent in fiscal
year 1999-2000.
SSA's Assessments Have Not Identified the Areas and Causes of Possible
Inconsistency in Decisions between Adjudication Levels:
Despite some efforts to assess inconsistency in decisions, shortcomings
in SSA's analyses also limit its ability to identify areas and causes
of possible inconsistency. Most notably, over the last 10 years, SSA
has not updated its prior analyses of information from its initial
biennial case review that helped identify problem areas. In addition,
SSA has not improved on its case review and analysis by ensuring that
reviewers assess all relevant case evidence used to make decisions, or
performed more sophisticated analysis to identify the areas and causes
of inconsistency in decisions. Other efforts--including the review of
ALJ allowances and a probe of DDS reconsideration denials--have yet to
yield useful information.
In 1994, for its initial biennial case review report, the agency took
its first step in identifying areas of possible inconsistency by
identifying two characteristics about the claimants and their cases
over which initial-level reviewers tended to disagree with ALJs.
Specifically, the 1994 report concluded that teams of reviewing medical
consultants and disability examiners sometimes viewed cases involving
mental impairments differently than the reviewing ALJs. In addition,
these two sets of reviewers tended to have different views on the
severity of claimants' impairments and their resulting capacity to
work. According to the official responsible for overseeing the review,
the findings in this initial report provided important support for
SSA's process unification efforts as well as the agency's efforts to
redesign the disability claims process.
SSA continues to conduct the biennial case reviews; however, the agency
has not continued to analyze and identify areas that are viewed
differently by different adjudication levels. Specifically, SSA no
longer identifies the particular case characteristics over which
reviewers from the two levels tend to disagree. As a result, SSA does
not know whether previously identified problem areas are still present.
Moreover, SSA no longer publishes any information from the medical
consultant and disability examiner biennial case reviews, even though
it has performed some limited analysis of the supportability of
decisions made by adjudicators. By not continuing to publish its
analysis and findings, the agency makes it difficult to ensure the
reliability of its methods and results, and leaves stakeholders outside
the agency, including disability groups, without a means for
understanding SSA's assessment efforts and progress in improving the
consistency of decisions. The SSA office conducting the study has told
us that, because of downsizing and competing priorities, it has no
current plans to further analyze and publish these data.
Further, in its ongoing biennial case reviews, SSA does not make full
use of available case information that would be useful in identifying
areas and causes of inconsistency. Specifically, medical consultants
and disability reviewers do not listen to tapes of the hearings and
therefore do not review the entire case as presented to the original
ALJ. Although reviewing medical consultants and disability examiners
read the ALJs' explanations for their original decisions, which should
include the most important factors behind the ALJs' decisions, the
reviewers do not evaluate the oral evidence independently. An SSA
official with whom we spoke indicated that some evidence entered by
witnesses at the hearing might not be accompanied by other hard copy
sources of the same information. Therefore, reviewers would not
consider information potentially relevant to the ALJ's decision that
could be used to identify areas and causes of inconsistency.
SSA also does not make full use of the information it collects because
it has not employed analytical tools that would improve its ability to
identify areas and causes of inconsistency. For example, SSA's biennial
case reviews provide a rich dataset that lends itself to regression
analysis to identify areas and possible causes of inconsistency between
levels. Regression analysis would allow the agency to better pinpoint
any significant case characteristics affecting decisions and to more
clearly identify the underlying causes of inconsistency. Specifically,
among the data collected in this review are such variables as the types
of impairments the claimant has, the types of relevant medical
evidence, and additional impairments presented at the hearing.
Multivariate analysis, such as a multiple regression model, could allow
SSA to assess how these and many other factors, relative to one
another, contribute to whether a case results in a similar outcome at
both levels. However, SSA has not employed this more sophisticated
multivariate technique, citing resource constraints and competing
priorities. We recognize the methodological complexities of analyzing
disability decisions, and we previously recommended that SSA establish
an advisory panel of external experts from a range of disciplines to
provide leadership, oversight, and technical assistance to the
agency.[Footnote 27] Otherwise, in forgoing such analysis, the agency
will continue to miss an opportunity to better pinpoint areas and some
possible causes of inconsistency in decisions between the two
adjudication levels, and to lay the foundation for further
investigation.
Another tool SSA has not sufficiently employed for identifying areas
and causes of inconsistency is in-depth case studies involving both
levels of adjudication. Case studies, in which different adjudicators
review the same test case, can be a means for unearthing causes for
inconsistency by getting adjudicators from both levels to acknowledge
and address discrepancies in the ways they view cases. SSA has
performed case studies in the past to ascertain differences in policy
interpretation between DDS examiners and quality reviewers. However,
SSA does not routinely have both DDS examiners and ALJs perform in-
depth review of the same sample of cases, despite this method's
potential for helping identify causes of inconsistency between the two
adjudication levels. OQA officials told us that case studies are a very
resource-intensive tool because they need a sufficient number of cases
from which to generalize. Therefore, the agency is reluctant to use
this approach to help it understand the causes of inconsistency between
adjudication levels. However, using multivariate analyses of the
biennial case review data could help the agency to more effectively
target its in-depth case studies on those areas found to be leading to
inconsistent decisions and thereby increase its success at identifying
the causes of inconsistency.
SSA conducts other analyses of inconsistency between levels, but to
date these efforts have yielded limited information concerning areas
and possible causes of inconsistency. For example, as part of SSA's ALJ
Pre-Effectuation Review, two different levels of reviewers have
evaluated thousands of cases. However, limitations in the review
methodology, such as not using a random sample of ALJ decisions, do not
allow the agency to use this review to identify the leading causes of
inconsistency. SSA recently began an evaluation of this effort and
plans to publish its findings and recommendations in April 2004.
Another analysis currently under way, a special 400-case review, might
help identify areas of inconsistency at the initial level, but it has
yet to be completed. Begun in 2003, this review by ALJs of DDS
reconsideration denial determinations is expressly aimed at assessing
inconsistency between adjudication levels. SSA expects to gain some
understanding of why about 60 percent of cases denied by the initial
level and appealed to the hearings level are allowed. The agency plans
to publish its findings in summer 2004.
SSA's New Proposal Incorporates Efforts to Improve the Consistency of
Decisions, but Challenges May Impede Successful Implementation:
Some changes included in SSA's new proposal to overhaul its disability
claims process may improve the consistency of DDS and ALJ decisions,
but challenges may hinder the implementation of the proposal. The new
proposal includes several changes to the disability claims process that
the agency and stakeholder groups representing adjudicators and
claimant representatives believe offer promise for improving the
consistency of DDS and ALJ decisions. However, past difficulties in
improving the process, as well as stakeholder concerns about limited
resources and other obstacles, indicate that some difficulties may
arise in the development and implementation of SSA's new proposal.
The Agency and Most Stakeholders Viewed Several Aspects of the New
Proposal as Offering Promise for Improving Consistency:
SSA told us that several aspects of the new proposal may improve the
consistency of decisions, and although opinions varied among
stakeholder groups, most thought the following four proposed changes
have the potential to improve the consistency of decisions between
adjudication levels: (1) requiring state adjudicators to more fully
develop and document their decisions, (2) centralizing the agency's
approach to quality control, (3) providing both adjudication levels
with equal access to more centralized medical expertise, and (4)
requiring ALJs to address agency reports that either recommend denying
the claim or outline the evidence needed to fully support the claim.
Representatives from most stakeholder groups with whom we spoke told us
that having state adjudicators more fully develop and document their
decisions may help to improve the consistency of DDS and ALJ decisions.
Specifically, stakeholders said that more developed decisions may
provide ALJs with a better understanding of the DDS decision and enable
them to more fully consider this information when evaluating a case.
According to the agency and stakeholders, this change may contribute to
a more consistent interpretation and application of SSA's decision-
making criteria. They also mentioned that well-developed decisions by
DDS examiners could assist SSA in holding adjudicators accountable for
case development and decisions, such as enabling quality reviewers to
more effectively assess the appropriateness of the DDSs decisions.
Unlike SSA's earlier attempt at more fully developing decisions as part
of process unification, SSA plans to incorporate a reviewing official
into the process whose assessment of all appealed DDS decisions can
provide feedback on the extent to which cases are being fully
developed.
In addition, the agency and many stakeholders told us that they believe
centralizing the agency's quality control system may help resolve some
problems contributing to inconsistent decisions between the two levels.
For example, they believed that it may help ensure a more consistent
review of cases across the country and between adjudication levels.
According to both stakeholders and other experts within and outside of
SSA (including SSA's Deputy Commissioner of Disability and Income
Security and a consulting group that reviewed SSA's quality assurance
system),[Footnote 28] the current quality control and case review
process encourages adjudicators at the initial level to inappropriately
deny cases, while encouraging adjudicators at the hearings level to
inappropriately allow cases. Specifically, by overemphasizing a review
of DDS allowances to help control the cost of benefits, the agency has
unintentionally encouraged DDS examiners to deny cases. Conversely,
SSA's review of ALJ decisions consists mostly of SSA's Appeals Council
reviewing cases denied by ALJs, thereby providing an incentive for ALJs
to allow cases. By centralizing the quality control system and making
other changes to the process, SSA believes that it can remove the
current incentives that contribute to inconsistency.
The third proposed change that the agency and most stakeholder groups
believe may improve consistency is SSA's plan to provide both
adjudication levels with equal access to more centralized medical
experts, organized by clinical specialty. Although located in the
regions, these experts should be able to review cases from across the
country with the successful completion of SSA's AeDib initiative--an
electronic folder initiative for exchanging case information currently
being implemented by SSA. By making experts in a range of specialties
available to assist both levels of adjudicators in their decision
making, SSA and stakeholders believe that adjudicators could more
consistently apply SSA's decision-making criteria, in addition to
acquiring better medical evidence.
Finally, the agency and most stakeholder groups told us that the
requirement to have an ALJ's decision address the recommendations from
a reviewing official's report to either deny or more fully develop the
claim may increase consistency between levels. Under the new proposal,
SSA plans to introduce a reviewing official into the process to
evaluate all appealed DDS claims. The official will allow claims that
meet SSA's definition of disability and, for the remaining claims, will
develop a report that either (1) contains reasons for denying the claim
or (2) outlines the evidence needed to fully support the claim. The
ALJ's decision must address issues raised in the reviewing official's
report. Stakeholders believed that this change could, as intended by
SSA, hold adjudicators more accountable for their decisions and provide
adjudicators with feedback on the reasons decisions tend to differ
between levels to improve the quality and consistency of their
decisions.
Although there was less agreement among stakeholder groups on the
potential effect that other aspects of the new proposal may have on the
consistency of decisions, some groups thought that other changes could
result in improved consistency between DDS and ALJ decisions. For
example, the Social Security Advisory Board and two groups representing
the DDSs thought that the proposed in-line quality control, if
implemented effectively at all levels, could have a positive impact on
consistency by ensuring that adjudicators adhere to the rulings and
regulations throughout the decision-making process. One stakeholder
group added that in-line quality control could also help the agency
identify problem areas, including areas in which policy is applied
inconsistently or where more training is needed.
Resource and Other Constraints May Limit SSA's Ability to Successfully
Implement Some Changes in the New Proposal:
According to stakeholder groups--and based on SSA's prior experience
with making significant changes to its claims process--insufficient
resources and other obstacles may prove to be major challenges for the
agency in developing and implementing aspects of its new proposal. For
example, experience with the process unification initiative has shown
that limited state resources have hindered the agency's ability to have
state adjudicators fully document decisions. To address this issue, SSA
plans to reduce the states' workloads by decreasing the number of
claims to be decided by the DDSs. Specifically, SSA expects that
establishing regional expert review units to make quick decisions for
claimants who are obviously disabled will substantially decrease the
states' workloads. However, SSA has not developed and provided
stakeholders with estimates of the administrative cost for more fully
documenting decisions and other planned changes, and stakeholder groups
were not convinced that the reduction in claims was sufficient to
offset resources needed to fully document their decisions. Although the
agency has had some recent success in increasing its 2004
administrative budget, and is confident that it will be successful in
acquiring the resources it needs to implement the proposal, the
significance of stakeholders' concerns about funding cannot be assessed
until SSA fully develops its proposal and associated cost estimates.
Experience has also shown that another proposed change, developing a
centralized quality control system for both adjudication levels, could
be a major challenge for the agency. In 1994, SSA began efforts to
create a unified and comprehensive quality control system as part of
its redesign efforts, but made little progress, in part because of
considerable disagreement among internal and external stakeholders on
how to accomplish this difficult objective. To get external assistance
in developing an effective quality assurance system, SSA contracted
with an independent consulting firm to assess SSA's quality assurance
practices used in the disability claims process. In 2001, concluding
that SSA could achieve its quality objectives for the disability
program only by adopting a broad, modern view of quality management,
the consulting firm recommended SSA abandon its current system and
design a new quality management system focused on building quality into
the process. The agency agreed that it was appropriate to transform the
existing quality assurance system and established an executive work
group to decide a future course of action. The agency is working with
another consulting group to further develop the changes recently
proposed by the Commissioner. However, after 10 years of efforts to
develop a more unified quality review system, SSA has not yet
formulated changes to its quality review system, beyond the brief and
general descriptions provided in the Commissioner's new proposal.
Other obstacles also add to the complexity and difficulty of
implementing the proposal. For example, stakeholder groups have raised
concerns about SSA's ability to successfully implement its proposed
change to provide equal access for all adjudicators to more centralized
medical expertise by removing medical expertise from the state DDSs and
providing expertise in regional offices. Stakeholder groups were
concerned that SSA would not be able to attract and retain sufficient
medical experts to meet the agency's needs. They told us that states
are currently experiencing problems attracting medical experts because
SSA's compensation rates are too low. State adjudicators, who currently
work with medical experts directly at DDS offices, were also concerned
that removing these experts and placing experts in SSA regional offices
would impair the states' effectiveness and efficiency. By placing
experts in regional offices, state disability examiners would no longer
have on-site access to these experts who help facilitate the states'
adjudication of claims and provide on-the-job training and mentoring to
DDS examiners.
Stakeholders have also raised questions about SSA's ability to ensure
that ALJs' decisions fully respond to the reviewing officials' reports
and the ultimate effectiveness of this change. Stakeholder groups
representing ALJs and claimant representatives believed that the
requirements may have the potential to impinge on an ALJ's legal
responsibility to ensure a claimant receives a fair hearing and an
independent decision. Other groups have raised concerns about SSA's
ability to ensure that ALJs will adequately address recommendations in
the reviewing officials' reports to help ensure that this requirement
leads to more consistent decisions. Although these concerns have been
raised, the Commissioner has clearly stated that the intent of the
proposal is to improve service to claimants, including providing fair
and accurate decisions, and that changes will not impinge on the
independence of ALJs.
In addition, several stakeholder groups also told us that staffing the
new reviewing official positions with attorneys, as SSA intends to do,
would be expensive. To the extent that SSA has difficulty filling these
positions, the agency could create a slowdown or bottleneck in the
process that could increase the time claimants must wait for a
decision. Furthermore, according to one stakeholder group, SSA's new
quality assurance process will need to ensure that this new position
does not create another source of inconsistent interpretation and
application of SSA's decision-making criteria.
Several groups representing hearings level adjudicators and claimant
representatives were also concerned about other aspects of the
Commissioner's new proposal, such as the proposed elimination of the
Appeals Council and the claimants' loss of the right to appeal an ALJ
decision to the council. The Appeals Council currently reviews about
100,000 appealed ALJ decisions annually. For these claims, the council
provides an additional appellate step for addressing claimants'
objections to the ALJs' decisions, reviewing new medical information on
the claims and reducing the number of claims appealed directly to the
federal courts. According to one stakeholder group, the council also
performs other important functions, such as reviewing claims for
surviving children or spouses of workers who were insured under the
disability insurance and retirement program. The council also reviews
cases remanded from federal courts. This stakeholder group also told us
that as SSA refines its proposal it will need to articulate how all of
the council's functions will be handled under the new process.
Adding to uncertainties about the proposal's success is its dependence
on the successful development and implementation of the AeDib system--
a highly complex and as yet unproven system using electronic folders to
share information with all entities involved in disability
determinations. SSA does not plan to implement its newly proposed
changes before it has completed a national rollout of its electronic
disability system, scheduled to be completed by October 2005. The new
electronic disability system represents an important step toward a
paperless and more efficient sharing of information by multiple
partners involved in the disability claims process, including SSA and
state officials, as well as physicians and other members of the medical
community who provide needed medical evidence. SSA also expects this
new system to provide critical management information for analyzing and
reducing inconsistencies in disability decisions. As we previously
reported, SSA has made progress developing the new system. However, its
approach involves risks that could jeopardize the agency's successful
transition to an electronic disability claims process.[Footnote 29] For
example, SSA recently began a national rollout of the electronic
disability system without fully evaluating pilot test results or
ensuring the resolution of all critical problems. Skipping such
important steps in development and implementation leaves the new system
vulnerable to problems in its performance and reliability. In addition,
problems with implementation of this system could delay the
implementation of SSA's new proposal.
SSA recognizes that transforming its massive and complex disability
programs and achieving the benefits envisioned by the Commissioner will
be a challenging undertaking. The agency is refining its proposal and,
as part of this process, is actively seeking input from stakeholder
groups. The Commissioner and her staff have met directly with
stakeholder groups to understand and begin to address their concerns.
As the agency refines its proposal, the significance of both
stakeholder concerns and previous problems SSA has experienced
improving its programs should become clearer.
Conclusions:
When SSA's Commissioner announced her new proposal to overhaul the
disability programs, the agency acknowledged the importance of making
similar decisions on similar cases and making the right decision as
early in the process as possible. SSA has good cause to focus on the
consistency of decisions between adjudication levels. Incorrect denials
at the initial level that are appealed increase both the time claimants
must wait for a decision and the cost of deciding cases. Incorrect
denials that are not appealed may leave needy individuals without a
financial or medical safety net. Conversely, incorrect allowances at
any adjudication level could substantially increase the cost of
providing disability benefits.
While the agency has made some effort to assess the inconsistency in
decisions between levels, its efforts have not provided the agency with
a clear understanding of the extent and leading causes of possible
inconsistencies in the interpretation and application of disability
guidance. For example, SSA's assessment of ALJ error-prone allowances
has not proven to be effective at identifying new areas and causes of
inconsistency. SSA also has not updated its more effective approach of
analyzing its Disability Hearings Quality Review data to identify
problem areas and help improve its understanding of the factors that
may be contributing to inconsistency. Further, SSA's analysis lacked
sophisticated statistical techniques and in-depth analysis of cases by
adjudicators at both levels, which together would have allowed SSA to
better identify and address the areas and leading causes of
inconsistency. Moreover, by not having examiners and medical
consultants perform a complete review of all relevant information
before an ALJ, SSA has limited its ability to understand the areas and
causes of possible inconsistency.
Without better information on the areas and causes of possible
inconsistency, the agency cannot ensure that the Commissioner's new
proposal will help to resolve this complex and long-standing concern.
By taking immediate actions to improve its understanding of the leading
causes of possible inconsistency in decisions, the agency will have
information needed to evaluate and possibly refine its new proposal,
including its plans to build an effective quality assurance system that
can both detect and prevent inconsistencies in decisions. This
information will help the agency to target its limited resources and
take decisive steps to build a claims process that provides claimants
with the accurate, consistent, and timely decisions they deserve, as
envisioned in the Commissioner's proposal.
Recommendations:
To move successfully forward with agency efforts to make more
consistent decisions, including efforts incorporated in the
Commissioner's proposal for an improved disability claims process and
quality assurance system, we recommend that SSA quickly expand its
assessment of the areas and causes of inconsistency in decisions
between adjudication levels. In doing so, SSA should consider making
near-term and cost-effective enhancements to its current approach for
assessing the consistency of decisions, including:
1. Reestablish ongoing analyses of case characteristics as part of its
biennial case review, in line with efforts undertaken for the review
report published in 1994.
2. Perform more sophisticated multivariate analysis on the biennial
case review data in order to pinpoint the most significant case
characteristics influencing allowance decisions and to distinguish
factors that might be contributing either appropriately or
inappropriately to allowance decisions.
3. Expand the biennial case review by requiring disability examiners
and medical consultants to review the hearing tapes to ensure that
reviewers have the complete case before them (including the types and
sources of testimonial evidence provided during the hearings) when
evaluating the ALJs' decisions.
4. Have adjudicators and reviewers from each level study cases in depth
to help pinpoint the causes of inconsistency, once potential areas of
inconsistency between levels are identified.
5. Publish the methods and findings of all analyses, to keep internal
and external stakeholders aware of the agency's efforts to assess
consistency and demonstrate improvement over time.
6. Use the information from these improved analyses to develop a more
focused and effective strategy for ensuring uniform application of
SSA's guidance and to improve the consistency of decisions. To
accomplish this, SSA should clarify guidance for making disability
decisions and develop mandatory training for adjudicators on issues
identified as contributing to inconsistency.
Agency Comments and Our Evaluation:
We provided a draft of this report to SSA for comment. SSA expressed
several reservations about the recommendations, findings, and
conclusions of our report. Primarily, SSA took issue with: (1) our
characterization of the agency's progress over the past several years
in analyzing and reducing the inconsistency of decisions, (2) our
recommendation that the agency incorporate multivariate analysis into
its assessments, and (3) our finding that the agency has not acted on
the results of its reviews of decisions. SSA indicated that it would
reevaluate our recommendations as the design of its Commissioner's new
approach to disability decision making evolves. However, the agency did
agree to pilot one recommendation--that quality reviewers assess
hearing tapes when evaluating the ALJs' decisions--as part of a quality
review.
One of SSA's main concerns was that our report did not fully discuss
the progress SSA had achieved in analyzing and reducing the
inconsistency in decision making between adjudication levels. For
example, SSA commented that our report dismissed the 21-percentage
point increase in the quality reviewers' support rate of ALJ decisions,
conducted as part of SSA's biennial case reviews over the last 10
years. SSA also pointed to findings from its ALJ peer reviews as
additional evidence that the quality and consistency of SSA's decisions
had improved. In addition, SSA asserted that its comparison of the
relative proportion of allowances at the DDS and ALJ levels, along with
high accuracy rates, indicated that adjudicators were making the right
decisions sooner in the process--a goal of both process unification and
the Commissioner's new disability approach. Although our report
incorporates results from the analyses cited by SSA, our conclusion
about the improvement in consistency between levels is not as
optimistic as SSA's because of weaknesses in SSA's assessments. As we
reported, SSA's analysis of the quality reviewers' assessment of ALJ
cases has been limited for 10 years to calculating ALJ support rates.
SSA has not used available data to determine the potential areas of
inconsistency between levels or the extent to which changes in the ALJ
support rate is related to improvements in consistency of decisions
between adjudication levels. SSA's assessment also lacks a reliable
method for determining whether DDS decisions are more consistent with
ALJ decisions, for example, by having ALJs regularly review a
statistical sample of DDS decisions. Lastly, as we pointed out, changes
in the proportion of overall allowances made by the DDS and ALJ levels
cannot serve as a reliable indicator for measuring the consistency of
decisions between levels, because many factors can affect these
proportions, such as significant fluctuations in the number of
decisions made at each adjudication level.
SSA also expressed its reservations about the benefits of multivariate
analysis in its evaluation of decision making. SSA asserted that its
analyses over the past 10 years have provided the agency with a solid
understanding of how certain variables influence disability decision
making and that the multivariate analyses we recommended would not
identify the causes and effects of inconsistent decision making at
different levels of this complex process. We agree with SSA that the
disability decision-making process is complex and that multivariate
analysis alone cannot establish all the causes and effects of
inconsistent decision making. However, because multivariate analysis
takes into account the influence of a number of relevant variables for
each decision, this analytical technique can provide a more accurate
understanding of areas and causes of inconsistency in decisions than
methods previously employed by SSA. Such analyses, followed by in-depth
case studies by adjudicators at both levels, which we also recommended,
would bring SSA closer to understanding and resolving the inconsistency
of decisions between adjudication levels. Therefore, we continue to
believe that by performing the analyses we recommend, the agency will
have a better understanding of the extent and causes of inconsistency,
and that SSA's Commissioner should quickly implement our
recommendations to ensure that her new approach effectively addresses
the consistency of decisions between adjudication levels.
Finally, SSA disagreed with our finding that it has not acted on the
results of its reviews of decisions. SSA noted that it has made changes
to address training needs that have been identified by its reviews.
Specifically, SSA indicated that it has provided a series of
interactive video training (IVT) sessions focusing on problematic areas
noted in the ALJ peer review reports. We acknowledge that SSA has
conducted ALJ peer reviews and used findings from its reviews to
develop and provide training to ALJs. However, we did not include these
findings in our report, because our objectives were limited to
reporting efforts undertaken by SSA to assess or improve the
consistency of decisions between adjudication levels or to implement
its process unification initiative. SSA's ALJ peer review is conducted
to identify problems with the quality of ALJ hearing process and
decisions, not to identify problems with the inconsistency of decisions
between levels. Conversely, our report included information on SSA's
ALJ pre-effectuation review, because it was part of SSA's process
unification initiative. According to information provided to us by SSA
during our audit, although this review was intended to help identify
policy and training areas that were associated with inconsistent
decisions between adjudication levels, it was not effective at
identifying any new areas to be pursued by the agency. This finding,
along with those provided throughout the report, supports our
recommendations to SSA that the agency perform additional analysis to
determine the causes of potential inconsistency between adjudication
levels and to clarify guidance and provide mandatory training to
address any identified causes.
In addition, SSA provided several other general and technical comments
about the draft report. These additional comments, as well as our
response to them, are provided in appendix II.
Copies of this report are being sent to the Commissioner of SSA,
appropriate congressional committees, and other interested parties. The
report is also available at no charge on GAO's Web site at
http://www.gao.gov. If you have any questions about this report, please
contact me at (202) 512-7215. Other contacts and staff acknowledgments
are listed in appendix III.
Sincerely yours,
Signed by:
Robert E. Robertson:
Director, Education, Workforce, and Income Security Issues:
[End of section]
Appendix I: Excerpt of SSA's Testimony Announcing Its New Proposal to
Improve Its Disability Decision-Making Process:
In designing my approach to improve the overall disability
determination process, I was guided by three questions the President
posed during our first meeting to discuss the disability
programs.[Footnote 30]
* Why does it take so long to make a disability decision?
* Why can't people who are obviously disabled get a decision
immediately?
* Why would anyone want to go back to work after going through such a
long process to receive benefits?
I realized that designing an approach to fully address the central and
important issues raised by the President required a focus on two over-
arching operational goals: (1) to make the right decision as early in
the process as possible; and (2) to foster return to work at all stages
of the process. I also decided to focus on improvements that could be
effectuated by regulation and to ensure that no SSA employee would be
adversely affected by my approach. My reference to SSA employees
includes State Disability Determination Service employees and
Administrative Law Judges (ALJs).
As I developed my approach for improvement, I met with and talked to
many people--SSA employees and other interested organizations,
individually and in small and large groups--to listen to their concerns
about the current process at both the initial and appeals levels and
their recommendations for improvement. I became convinced that
improvements must be looked at from a system-wide perspective and, to
be successful, perspectives from all parts of the system must be
considered. I believe an open and collaborative process is critically
important to the development of disability process improvements. To
that end, members of my staff and I visited our regional offices, field
offices, hearing offices, and State Disability Determination Services,
and private disability insurers to identify and discuss possible
improvements to the current process.
Finally, a number of organizations provided written recommendations for
changing the disability process. Most recently, the Social Security
Advisory Board issued a report prepared by outside experts making
recommendations for process change. My approach for changing the
disability process was developed after a careful review of these
discussions and written recommendations. As we move ahead, I look
forward to working within the Administration and with Congress, as well
as interested organizations and advocacy groups. I would now like to
highlight some of the major and recurring recommendations made by these
various parties.
The need for additional resources to eliminate the backlog and reduce
the lengthy processing time was a common theme. This important issue is
being addressed through my Service Delivery Plan, starting with the
President's FY 2004 budget submission which is currently before
Congress. Another important and often heard concern was the necessity
of improving the quality of the administrative record. DDSs expressed
concerns about receiving incomplete applications from the field office;
ALJs expressed concerns about the quality of the adjudicated record
they receive and emphasized the extensive pre-hearing work required to
thoroughly and adequately present the case for their consideration. In
addition, the number of remands by the Appeals Council and the Federal
Courts make clear the need for fully documenting the administrative
hearing record.
Applying policy consistently in terms of: 1) the DDS decision and ALJ
decision; 2) variations among state DDSs; and 3) variations among
individual ALJs--was of great concern. Concerns related to the
effectiveness of the existing regional quality control reviews and ALJ
peer review were also expressed. Staff from the Judicial Conference
expressed strong concern that the process assure quality prior to the
appeal of cases to the Federal Courts.
ALJs and claimant advocacy and claimant representative organizations
strongly recommended retaining the de novo hearing before an ALJ.
Department of Justice litigators and the Judicial Conference stressed
the importance of timely case retrieval, transcription, and
transmission. Early screening and analysis of cases to make expedited
decisions for clear cases of disability was emphasized time and again
as was the need to remove barriers to returning to work.
My approach for disability process improvement is designed to address
these concerns. It incorporates some of the significant features of the
current disability process. For example, initial claims for disability
will continue to be handled by SSA's field offices. The State
Disability Determination Services will continue to adjudicate claims
for benefits, and Administrative Law Judges will continue to conduct
hearings and issue decisions. My approach envisions some significant
differences.
I intend to propose a quick decision step at the very earliest stages
of the claims process for people who are obviously disabled. Cases will
be sorted based on disabling conditions for early identification and
expedited action.
Examples of such claimants would be those with ALS, aggressive cancers,
and end-stage renal disease. Once a disability claim has been completed
at an SSA field office, these Quick Decision claims would be
adjudicated in Regional Expert Review Units across the country, without
going to a State Disability Determination Service. This approach would
have the two-fold benefit of allowing the claimant to receive a
decision as soon as possible, and allowing the State DDSs to devote
resources to more complex claims.
Centralized medical expertise within the Regional Expert Review Units
would be available to disability decision makers at all levels,
including the DDSs and the Office of Hearings and Appeals (OHA). These
units would be organized around clinical specialties such as
musculoskeletal, neurological, cardiac, and psychiatric. Most of these
units would be established in SSA's regional offices.
The initial claims not adjudicated through the Quick Decision process
would be decided by the DDSs. However, I would also propose some
changes in the initial claims process that would require changes in the
way DDSs are operating. An in-line quality review process managed by
the DDSs and a centralized quality control unit would replace the
current SSA quality control system. I believe a shift to in-line
quality review would provide greater opportunities for identifying
problem areas and implementing corrective actions and related training.
The Disability Prototype would be terminated and the DDS
Reconsideration step would be eliminated. Medical expertise would be
provided to the DDSs by the Regional Expert Review units that I
described earlier.
State DDS examiners would be required to fully document and explain the
basis for their determination. More complete documentation should
result in more accurate initial decisions. The increased time required
to accomplish this would be supported by redirecting DDS resources
freed up by the Quick Decision cases being handled by the expert units,
the elimination of the Reconsideration step, and the shift in medical
expertise responsibilities to the regional units.
A Reviewing Official (RO) position would be created to evaluate claims
at the next stage of the process. If a claimant files a request for
review of the DDS determination, the claim would be reviewed by an SSA
Reviewing Official. The RO, who would be an attorney, would be
authorized to issue an allowance decision or to concur in the DDS
denial of the claim. If the claim is not allowed by the RO, the RO will
prepare either a Recommended Disallowance or a Pre-Hearing Report. A
Recommended Disallowance would be prepared if the RO believes that the
evidence in the record shows that the claimant is ineligible for
benefits. It would set forth in detail the reasons the claim should be
denied. A Pre-Hearing Report would be prepared if the RO believes that
the evidence in the record is insufficient to show that the claimant is
eligible for benefits but also fails to show that the claimant is
ineligible for benefits. The report would outline the evidence needed
to fully support the claim. Disparity in decisions at the DDS level has
been a long-standing issue and the SSA Reviewing Official and creation
of Regional Expert Medical Units would promote consistency of decisions
at an earlier stage in the process.
If requested by a claimant whose claim has been denied by an RO, an ALJ
would conduct a de novo administrative hearing. The record would be
closed following the ALJ hearing. If, following the conclusion of the
hearing, the ALJ determines that a claim accompanied by a Recommended
Disallowance should be allowed, the ALJ would describe in detail in the
written opinion the basis for rejecting the RO's Recommended
Disallowance. If, following the conclusion of the hearing, the ALJ
determines that a claim accompanied by a Pre-Hearing Report should be
allowed, the ALJ would describe the evidence gathered during the
hearing that responds to the description of the evidence needed to
successfully support the claim contained in the Pre-Hearing Report.
Because of the consistent finding that the Appeals Council review adds
processing time and generally supports the ALJ decision, the Appeals
Council stage of the current process would be eliminated. Quality
control for disability claims would be centralized with end-of-line
reviews and ALJ oversight. If an ALJ decision is not reviewed by the
centralized quality control staff, the decision of the ALJ will become
a final agency action. If the centralized quality control review
disagrees with an allowance or disallowance determination made by an
ALJ, the claim would be referred to an Oversight Panel for
determination of the claim. The Oversight Panel would consist of two
Administrative Law Judges and one Administrative Appeals Judge. If the
Oversight Panel affirms the ALJ's decision, it becomes the final agency
action. If the Panel reverses the ALJ's decision, the oversight Panel
decision becomes the final agency action. As is currently the case,
claimants would be able to appeal any final agency action to a Federal
Court.
At the same time these changes are being implemented to improve the
process, we plan to conduct several demonstration projects aimed at
helping people with disabilities return to work. These projects would
support the President's New Freedom Initiative and provide work
incentives and opportunities earlier in the process.
Early Intervention demonstration projects will provide medical and cash
benefits and employment supports to Disability Insurance (DI)
applicants who have impairments reasonably presumed to be disabling and
elect to pursue work rather than proceeding through the disability
determination process. Temporary Allowance demonstration projects will
provide immediate cash and medical benefits for a specified period (12-
24 months) to applicants who are highly likely to benefit from
aggressive medical care. Interim Medical Benefits demonstration
projects will provide health insurance coverage to certain applicants
throughout the disability determination process. Eligible applicants
will be those without such insurance whose medical condition is likely
to improve with medical treatment or where consistent, treating source
evidence will be necessary to enable SSA to make a benefit eligibility
determination. Ongoing Employment Supports to assist beneficiaries to
obtain and sustain employment will be tested, including a Benefit
Offset demonstration to test to effects of allowing DI beneficiaries to
work without total loss of benefits by reducing their monthly benefit
$1 for every $2 of earnings above a specified level and Ongoing Medical
Benefits demonstration to test the effects of providing ongoing health
insurance coverage to beneficiaries who wish to work but have no other
affordable access to health insurance.
I believe these changes and demonstrations will address the major
concerns I highlighted earlier. I also believe they offer a number of
important improvements:
* People who are obviously disabled will receive quick decisions.
* Adjudicative accountability will be reinforced at every step in the
process.
* Processing time will be reduced by at least 25%.
* Decisional consistency and accuracy will be increased.
* Barriers for those who can and want to work would be removed.
Describing my approach for improving the process is the first step of
what I believe must be--and will work to make--a collaborative process.
I will work within the Administration, with Congress, the State
Disability Determination Services and interested organizations and
advocacy groups before putting pen to paper to write regulations. As I
said earlier, and I say again that to be successful, perspectives from
all parts of the system must be considered.
Later today, I will conduct a briefing for Congressional staff of the
Ways and Means and Senate Finance Committees. I will also brief SSA and
DDS management. In addition, next week I will provide a video tape of
the management briefing describing my approach for improvement to all
SSA regional, field, and hearing offices, State Disability
Determination Services, and headquarters and regional office employees
involved in the disability program. Tomorrow, I will be conducting
briefings for representatives of SSA employee unions and interested
organizations and advocacy groups, and I will schedule meetings to
provide an opportunity for those representatives to express their views
and provide assistance in working through details, as the final package
of process improvements is fully developed.
I believe that if we work together, we will create a disability system
that responds to the challenge inherent in the President's questions.
We will look beyond the status quo to the possibility of what can be.
We will achieve our ultimate goal of providing accurate, timely service
for the American people.
[End of section]
Appendix II: Comments from the Social Security Administration:
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
SOCIAL SECURITY:
The Commissioner:
June 4, 2004:
Mr. Robert E. Robertson:
Director, Education, Workforce and Income Security Issues:
U.S. General Accounting Office:
Room 5-T-57:
441 G Street, NW:
Washington, D.C. 20548:
Dear Mr. Robertson:
Thank you for the opportunity to review and comment on the draft report
"Social Security Administration --More Effort Needed to Assess
Consistency of Disability Decisions" (GAO-04-656). Our comments on the
report are enclosed.
If you have any questions, please have your staff contact Candace
Skumik, Director, Audit Management and Liaison Staff, at (410) 965-
4636.
Sincerely,
Signed by:
Jo Anne B. Barnhart:
Enclosure:
SOCIAL SECURITY ADMINISTRATION BALTIMORE MD 21235-0001:
COMMENTS ON THE GENERAL ACCOUNTING OFFICE (GAO) DRAFT REPORT "SOCIAL
SECURITY ADMINISTRATION --MORE EFFORT NEEDED TO ASSESS CONSISTENCY OF
DISABILITY DECISIONS" (GAO-04-656):
Thank you for sharing the draft report with us.
We have several reservations about the findings, conclusions and
recommendations of this report. Our overall impression is that GAO has
focused on the shortcomings of "process unification," but is hesitant
to fully discuss the progress that the Social Security Administration
(SSA) has achieved in not only investigating and analyzing the
underlying causes of inconsistent decision-making between the State
disability determination service (DDS) examiners and Administrative Law
Judges (ALJs), but also the Agency efforts to deal with the problem.
For example, the draft report frequently notes that "SSA's assessments
have not provided a clear understanding on the extent and causes of
possible inconsistencies in decisions between adjudication levels." We
believe that the data obtained and analyzed over the last 10 years have
supplemented and further added to the findings and conclusions of the
original 1994 Disability Hearings Quality Review Process (DHQRP)
Report.
When discussing the data provided by SSA's reports, the GAO report
seems to have overlooked some of the key findings in these reports. For
example, although GAO acknowledged that the medical consultant/
disability examiner (MC/DE) review showed an improved hearing allowance
support rate of 57 percent in fiscal years (FY) 2001-2002 over the FYs
1993-1994 support rate of 36 percent, it dismisses this improvement as
portraying only a partial picture. However, a 21-percentage point
increase over this period of time is tremendous - especially given the
fact that it occurred during a period of time in which the hearing-
level allowance rate increased. Moreover, findings of the peer review
process conducted by detailed ALJs, i.e., reviewing judges (RJ),
confirm that allowance quality is on the increase. More importantly,
for process unification purposes, the "gap" between these MC/DE and RJ
support rates is narrowing, indicating that the process unification
initiatives have resulted in improved decisional accuracy and increased
compliance with Agency program policy.
Disability decision-making is a complex process at the DDS level, and
even more so at the hearing level given the potential impact of
testimony and a de novo hearing. The process takes into account both
objective and subjective factors-and all cannot be quantified. As such,
running the multivariate analyses recommended by GAO would not
establish the total cause and effect they seek in trying to solve the
enigma of inconsistent decision-making at different levels. Over the
past 10 years, the Agency has conducted several analyses that take into
account case characteristics. While additional studies can be
conducted, we believe that these analyses have provided the Agency with
a solid understanding of how certain variables influence disability
decision-making.
We are disappointed in GAO's view that our approach to tracking annual
allowance rates and trends is "simplistic and inconclusive" because it
uses "snapshot" data rather than tracking cohorts of cases through the
appeals process. Although tracking a cohort of cases is a viable
alternative, we disagree with the GAO assessment of our approach. We
believe our data are representative of each year analyzed; i.e., about
6 years of disability dispositions. As such, there were no DDS and
Office of Hearings and Appeals (OHA) disability dispositions
representing a different set of demographics and case characteristics
during that period of time that we failed to include in our analyses.
Given the sheer volume of cases adjudicated by both the DDSs and OHA
during a relatively stable economic period, we believe our findings
provided an accurate portrayal of the allowance rate dynamic (see table
below). Not only were DDSs allowing more cases, but their accuracy was
high as well. This is indicative of adjudicators making the right
decision sooner in the process-a goal of both process unification and
the Commissioner's new disability approach.
[See PDF for table]
[End of table]
One of GAO's recommendations calls for expanding the DHQRP to permit
the MC/DEs to review the hearing tape as part of their ongoing review
process. Earlier, we had indicated to GAO that resources would be a
concern as review time could be increased by about 45 minutes per
reviewer (e.g., if one DE and three MCs review a case, the additional
time would equal 3 hours). However, as a compromise, we propose
conducting a test in which the DEs audit the hearing tape. As such, we
would gain the benefit of the DE's assessment at a much smaller
resource cost. We would then report our findings and conclusions
concerning whether the DE audit would impact the outcome of the review
process.
We are also concerned about comments in the report regarding the
failure to make changes as a result of findings from reviews of the
decisions. On page 15, the report states, "As part of the initiative,
the agency also implemented a review of judges' allowances decisions to
identify additional ways to improve training and policies, but no new
changes were made as a result of findings from the review." On page 19,
the report states, "However, according to a SSA official, this review
did not identify any new areas of inconsistency that required
improvements to policy and training." And the final recommendation of
the report suggests that SSA should "clarify guidance for making
disability decisions and develop mandatory training for adjudicators on
issues identified as contributing to inconsistency." We disagree with
the assertion that SSA has not addressed training needs that have been
identified as the result of these reviews. In fact, SSA provides a
series of Interactive Video Training (IVT) sessions focusing on
problematic areas noted in the ALJ Peer Review Report. These broadcasts
have occurred bi-monthly, cover a broad range of topics and all ALJs
are required to view them, either during a live broadcast or by viewing
a videotape of the broadcast at a later date.
On September 25, 2003, the Commissioner testified before the House Ways
and Means Subcommittee on Social Security and presented her approach to
improve the disability determination process. She explained that in
designing the approach, she listened to a number of interested parties
who expressed concerns about applying policy consistently in terms of
(1) the DDS decision and ALJ decision; (2) variations among the state
DDSs; and (3) variations among individual ALJs. A main theme of the new
approach is to make the right decision as early in the process as
possible. We are pleased with GAO's recognition that several aspects of
the new approach are intended to improve the accuracy, timeliness, and
consistency of decisions, such as having the DDS decision-makers more
fully develop and document their decisions, providing for centralized
quality review of all decisions, and providing all adjudication levels
equal access to medical expertise. The new approach takes a
comprehensive look at quality at each step of the disability
adjudication process. At its core, the approach includes an in-line
quality assurance process and a centralized quality control review that
will replace the current SSA quality assurance system. This shift to an
in-line quality assurance review will provide greater opportunities for
identifying problem areas and implementing corrective actions and
related training. Likewise, a centralized quality control review will
provide end-of-line reviews and timely feedback to disability decision-
makers.
The main theme of the GAO report is that in the current disability
process, the DDS examiners and the ALJ's may be interpreting SSA's
criteria differently. This inconsistency was also a major theme in the
comments that were considered in designing the new approach. A key
aspect of the approach is the creation of feedback loops between each
stage of the disability process that are intended to increase
accountability between decisional levels and reduce decisional
inconsistency. For example, ALJ's will be expected to provide feedback
to reviewing officials, and reviewing officials will likewise provide
feedback to the DDSs. Meanwhile, the DDSs will have their own in-line
quality assurance process that will provide consistency within each
State entity. In short, we will be ensuring quality and decisional
consistency at all levels of the disability determination process by
having "downstream" adjudicators provide essential feedback and
management information to earlier "upstream" adjudicators.
However, we would like to clarify several aspects of the new approach
that are described in the GAO report. The report notes that "providing
both adjudication levels with equal access to more centralized medical
expertise" could improve decisional consistency. The new approach
relies on ensuring availability of enhanced medical expertise to all
levels of adjudication - DDS examiners, Reviewing Officials, ALJs and
Oversight Panels. It is critically important that a medical expert with
the appropriate expertise be available to assist the adjudicators.
However, no final decision has been made concerning the locations of
these experts. With e-Dib in place, access to the appropriate experts
will be available, regardless of location.
The GAO report also notes that decisional consistency will be improved
by "requiring ALJs to address agency reports that either recommend
denying the claim or outline the evidence needed to fully support the
claim." We believe that decisional consistency and improved quality
will be achieved only if greater accountability is established at each
stage of the disability process. As noted earlier, the new approach to
disability determinations provides for feedback at all levels of
adjudication. For example, although we expect that e-Dib, and
especially implementation of the Electronic Disability Collect System
(EDCS), will greatly enhance the information received by the DDS
examiners from the SSA field office (FO), if the quality of this
information is not adequate, we expect feedback from the DDS to the
SSA. FO. Similarly, if the Reviewing Official (RO) does not find that
the DDS examiner's explanation of the determination is adequate, the RO
will remand the case to the DDS to ensure that the DDS provides
adequate documentation for the basis of the decision. If the RO
disagrees with the determination of the DDS examiner, the RO will be
expected to explain to the DDS examiner the reasons that the RO reached
a different decision. Of course, if the claimant requests a hearing,
the ALJ will be issuing a de novo decision. While the RO decision is
not controlling and will be accorded no weight by the ALJ, if the ALJ
disagrees with the RO decision, the ALJ will be expected to give
feedback to the RO by addressing the matter in the decision. This
feedback will be crucial to ensuring that the RO understands the basis
for the ALJ disagreement so that subsequent evaluations by the RO will
be more responsive. The improved documentation requirements and the
feedback loops at each stage of the process are designed to ensure that
each decision maker is consistently interpreting and applying SSA
criteria. This enhanced accountability at all stages of the process
will be critical to ensure improved decisional consistency.
And while the GAO report portrays the implementation of an electronic
disability system as a vulnerability to the implementation of the new
disability approach, it is also important to recognize the value of the
system as an essential tool that will allow SSA to capture critical
management information needed to reduce decisional variance. Analyzing
this management information will help determine whether decisional
variance results from human error, overly complex policies, or faulty
training. Thus, either critical training resources can be applied where
needed or complex policies that might be driving inconsistent decision
making can be simplified. In short, this electronic management
information model will provide a wealth of opportunities to highlight
and remedy pockets of decisional variance at all levels.
As we move forward, we will consider GAO's recommendations vis-a-vis
methodology changes for quality measures, as appropriate. We are deeply
committed to improving both the quality of and the consistency within
disability decision-making across SSA's adjudicatory process. However,
the recommendations in the report will need to be revaluated as the
design of the new disability approach matures.
We have the following technical comments on the report:
* To avoid any confusion, we suggest using the term ALJ instead of
judge throughout the report, unless referring to a district court
judge.
* On page 7, Figure 1, the chart apparently was derived from an SSA
document; the fiscal year (FY) 2003 Waterfall Chart prepared by Agency
staff for GAO. It omits the footnote that explains that the data is not
longitudinal. Because the chart is not longitudinal, using a cohort of
claims, the GAO methodology produces results that are very different
from those experienced in the actual disability determination process
and subjects the results to the same critique GAO makes of our failure
to follow a cohort. We would suggest replacing the chart with the cross
sectional waterfall we provide Congress for their "Green Book."
* Also, the chart (page 7, Figure 1) fails to note that about 25
percent of the initial claims workload is subject to an alternate
process that does not include the reconsideration step, which affects
the appellate level data in the chart. Footnote #1 from the FY 2003
Waterfall Chart would address those issues.
* The first full sentence at the top of page 12 is unclear. We suggest
using two sentences: "Instead, studies indicate that systemic
differences in the assessment of claims at both adjudication levels are
contributing to the ALJ allowance rate. An example of one such study is
our 1997 report that noted that differences in the views held by state
DDSs and ALJs regarding a claimant's functional abilities was a key
factor explaining why ALJs allowed cases on appeal."
* On page 12, footnote #11, is misleading. We believe it is more
appropriate to read as follows: "The overall Agency cost, both Federal
and DDS costs, for an average initial claim decision was about $812,
while...."
GAO Comments:
1. We maintain that our report fully and fairly describes SSA's
progress in analyzing and addressing the underlying causes of
inconsistent decisions between state DDS examiners and ALJs. Our
research included an extensive review of agency documentation and
interviews with SSA officials, as well as stakeholder groups for
adjudicators and claimant representatives, to develop a complete
understanding of the agency's efforts to assess and improve the
consistency of decisions between adjudication levels. Also, in
agreement with our requestor, we sought to expand the review to include
SSA's new approach to improving its disability programs, so that we
could provide the Congress with an understanding of how SSA's future
plans may help to address this issue.
2. We provided information on the various reviews and analyses of
disability decisions to assess the consistency of decisions between
adjudication levels conducted by SSA over the last 10 years, but none
of these reviews have clearly identified the causes of inconsistency in
decisions between adjudication levels.
3. Our report has not overlooked the data cited by SSA. Nevertheless,
our conclusion about the improvement in consistency between levels
indicated by the data is not as optimistic as SSA's because of
weaknesses in SSA's assessments. As we reported, for 10 years SSA's
analysis of the quality reviewers' assessment of ALJ cases has been
limited to calculating ALJ support rates. SSA has not used available
data to determine the potential areas of inconsistency between levels
nor the extent to which changes in the ALJ support rate are related to
improvements in consistency of decisions between adjudication levels.
SSA's assessment only provides a general indication of overall changes
in consistency at one adjudication level.
4. Our report recognizes that SSA's disability decision-making process
is complex. Because of this complexity, we believe that multivariate
analysis is an appropriate assessment tool that would allow SSA to
assess the effect of multiple factors. In recommending this
sophisticated tool, we were careful not to imply that causes and
effects of inconsistent decision making can be established with
certainty. However, we believe that such an analysis will help SSA
understand the relative importance of the variety of factors that
affect its decision-making process. After identifying areas of
inconsistency, SSA can target these areas with in-depth case analyses
to pinpoint the causes of inconsistency and develop a more effective
strategy for addressing inconsistency. On the basis of our review of
SSA's analyses to date, we do not agree with the implications of SSA's
comments that it has a solid understanding of how certain variables
influence disability decision making, and therefore does not need to
conduct additional, more sophisticated analyses.
5. We agree with SSA that the proportion of allowances made at each
level can provide some insight into the allowance rate dynamic.
However, as we reported, we do not believe that it can serve as a
reliable indicator of the agency's progress in achieving more
consistent decisions between the DDS and OHA levels. The allowance data
provided by SSA simply show that the relative proportion of allowances
made at the DDS level increased in comparison with the OHA level, but
SSA has not performed any additional analysis to show that these
changes have any relationship to improved consistency in decision
making between the two adjudication levels. Additional analysis is
needed because a myriad of factors, such as changes in the economy, can
affect allowance rates. Although SSA claims that over this period of
time the economy has been "relatively stable," without performing any
additional analysis it cannot eliminate changes in the economy or
demographics of claimants as an influence on the allowance rates at
each level. In addition, SSA has not analyzed how other factors, such
as changes in productivity and total number of decisions made at each
level, may be influencing the allowance data.
6. The allowance rate data provided by SSA in its comments is very
similar to that provided by SSA earlier to us and included in our
report in figure 2. The figures we reported for the proportion of
allowances made by the DDS and OHA levels for fiscal years 1997 and
1998 vary in comparison with those provided by SSA by one percentage
point. We have not changed the figures in our report because we believe
that these slight differences simply reflect that we reported data
based upon fiscal, not calendar, years.
7. In our report, our statements that SSA has not made changes as a
result of findings from its reviews were specifically related to SSA's
ALJ pre-effectuation review. We included information on this review
because it was part of SSA's process unification initiative and was
intended to identify policy and training areas associated with
inconsistent decisions between adjudication levels. During our review,
we were told by an SSA official that the ALJ pre-effectuation review
was not successful at identifying new areas of inconsistency to be
addressed by SSA. In its comments, SSA cites a review unrelated to
assessing the inconsistency of decisions between levels, the ALJ peer
review, to assert that it has used reviews to identify training issues
to improve the quality of decisions. The lack of success with the ALJ
pre-effectuation review--along with other findings showing a limited
understanding of the cause of inconsistency--supports our
recommendations to SSA to perform additional analysis and to clarify
guidance and provide mandatory training to address any identified
causes of inconsistency between adjudication levels.
8. We applaud SSA's plans to use the electronic disability system to
capture critical management information to address decisional variance
or inconsistency, which could provide a wealth of useful information
for the agency. We have adjusted our report's text to reflect this
additional purpose. We continue to believe that SSA should not wait for
the development of this system, but should proceed to perform
multivariate analysis, using available data from its biennial case
reviews, to start identifying areas of potential inconsistency between
adjudication levels.
9. We applaud SSA's deep commitment to improving the disability
decision-making process, but believe that additional efforts to
understand the causes of potential inconsistencies in decision making
would help to inform the design of the Commissioner's new approach and
should, therefore, be undertaken immediately.
10. We generally agree with the technical comments provided and changed
the text accordingly.
[End of section]
Appendix III: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Michele Grgich, Assistant Director (415) 904-2183 Julie DeVault,
Analyst in Charge (415) 904-2269:
Staff Acknowledgments:
In addition to the individuals mentioned above, the following staff
members made major contributions to this report: Michael Morris,
Corinna Nicolaou, Walter Vance, and Rebecca Woiwode. Douglas Sloane
provided assistance with methodological issues, and Daniel Schwimer
provided legal support.
FOOTNOTES
[1] The figures include federal payments for the Disability Insurance
and the Supplemental Security Income programs to beneficiaries who have
a disability or are blind and their families.
[2] For additional information on the major problems facing federal
disability programs and our decision to designate them as high-risk,
see U.S. General Accounting Office, High-Risk Series: An Update,
GAO-03-119, (Washington, D.C.: January 2003).
[3] We took additional steps to assess the reliability of data included
in our report. For the proportion of initial and hearings levels
allowances, we verified SSA's calculations of summary data from its
workload reporting systems. For the information from the Disability
Hearings Quality Review, we reviewed the weights and calculations used
by SSA to determine national support rates for reviewed decisions and
found them to be correctly calculated and reliable.
[4] These groups included the National Association of Disability
Examiners, the National Council of Disability Determination Directors,
the Association of Attorney Advisors, the Association of Administrative
Law Judges, the National Treasury Employee Union, the Public Employees
Federation, the Social Security Section of the Federal Bar Association,
and the National Organization of Social Security Claimants'
Representatives.
[5] The figures include federal payments for the Disability Insurance
and the Supplemental Security Income programs to beneficiaries who have
a disability or are blind and their families. The figures were
calculated based on statistical information from SSA's web site.
[6] These figures are based on information provided on SSA's web site
from the 2004 OASDI Trustees Report, Part III. Financial Operations of
the Trusts Funds and Legislative Changes in the Last Year.
[7] Others who may testify at ALJ hearings frequently include
vocational and medical experts.
[8] Substantial gainful work is a level of work activity that involves
doing significant physical or mental work, or a combination of both,
that is productive. SSA has established earning criteria as a
reasonable indication of whether claimants are able to engage in
substantial gainful activity. In 2004, SSA generally considered
claimants to be engaging in substantial gainful activity if their
earnings averaged over $810 a month.
[9] See Implementation of Section 304 (g) Public Law 96-265, Social
Security Disability Amendments of 1980 (the Bellmon Report), Secretary
of Health and Human Services (Washington, D.C.: January 1982).
[10] Findings from SSA's 1994 Disability Hearings Quality Review Process
report provided the agency with additional information on potential
causes of inconsistency. The report identified two assessment areas
associated with inconsistent decisions. In addition, quality reviewers
found that when applying standards used by the initial level to
adjudicate claims, 29 percent of the appealed DDS reconsideration
decisions and 51 percent of ALJ decisions were not supported by the
decision makers. These findings help to support SSA's decision to
include efforts to have DDSs more fully develop and document their
decisions and to assess ALJ decisions as part of its process
unification initiative.
[11] U.S. General Accounting Office, Social Security Disability: SSA
Must Hold Itself Accountable for Continued Improvement in Decision-
Making, GAO/HEHS-97-102 (Washington, D.C.: Aug. 12, 1997).
[12] The board also identified factors that could affect consistency
within adjudication levels, as opposed to strictly between adjudication
levels. For more information see the Social Security Advisory Board,
Disability Decision Making: Data and Materials, January 2001, pp. 5-6.
[13] An appeal adds significantly to costs associated with making a
decision. According to SSA's Performance and Accountability Report for
fiscal year 2001, the average cost per claim for an initial DDS
disability decision was about $583, while the average cost per claim of
an ALJ decision was an estimated additional $2,157.
[14] An appeal also significantly increases the time required to reach
a decision. According to SSA's Performance and Accountability Report
for fiscal year 2003, the average number of days that claimants waited
for an initial decision was 97 days, while the number of days they
waited for an appealed decision was 344 days. However, the time a
claimant waits for a decision should not impact the amount of benefits
received, as benefits are based on the date the claimant becomes
disabled.
[15] U.S. General Accounting Office, Performance and Accountability
Series: Major Management Challenges and Program Risks: Social Security
Administration, GAO-03-117 (Washington, D.C.: January 2003).
[16] Social Security Ruling 96-2: Policy Interpretation Ruling Titles
II and XVI: Giving Controlling Weight to Treating Source Medical
Opinions, effective July 2, 1996.
[17] Since SSA rulings are binding only on SSA adjudicators and do not
have to be followed by the courts, SSA planned to strengthen the impact
of several rulings by creating regulations that would be followed by
the courts.
[18] These guidelines--called the Program Operations Manual System
(POMS)--contain, within an estimated 30,000 pages, interpretations of
relevant statutes, regulations, and rulings and procedural information.
[19] Additional DDS and OHA sites may have taped and viewed this
training, but SSA's monitoring of training provides only the number of
sites logged on to the interactive video training.
[20] U.S. General Accounting Office, Social Security Administration:
Strategic Workforce Planning Needed to Address Human Capital Challenges
Facing the Disability Determination Services, GAO-04-121 (Washington,
D.C.: Jan. 27, 2004).
[21] When reviewing an ALJ allowance, OQA uses the preponderance of
evidence standard, which requires that the reviewer fairly consider all
evidence and decide whether the weight of that evidence supports the
allowance.
[22] The Appeals Council uses a substantial evidence standard of review
that requires the reviewer to determine that the evidence in the case
is sufficient to convince a reasonable mind of the credibility of the
allowance decision, and that there is no opposing evidence that clearly
compels another finding or conclusion.
[23] At the hearings level, ALJs can purchase medical evidence, at
their discretion.
[24] See the Social Security Administration, Office of Program and
Integrity Reviews, Findings of the Disability Hearings Quality Review
Process: An Assessment of the Quality of Hearing Decisions and Appealed
Reconsideration Denial Determinations, September 1994.
[25] An OQA official also said that results from another review bolster
the conclusion that the quality of ALJ decisions has improved.
Specifically, in the ALJ peer review, in which ALJs evaluate a random
sample of other ALJs' decisions, the reviewing ALJs have found an
increasing percentage of ALJ decisions to be supportable. The percent
of decisions found to be supportable increased from 81 percent to 90
percent from the reviews of decisions issued from fiscal years 1993
through 1994 compared with decisions from fiscal years 2001 through
2002.
[26] See the Social Security Administration, Office of Quality
Assurance and Performance Assessment, Quality Assurance Group I Initial
Denial Accuracy Report: August-October 2002, July 2003.
[27] In addition, the report recommended that SSA include cases
appealed to its Appeals Council in the sample for its biennial case
review to eliminate the systematic bias in that sample and make it
representative of all cases that receive a decision from the hearings
level. For more information on this recommendation and others, see U.S.
General Accounting Office, SSA Disability Decision Making: Additional
Steps Needed to Ensure Accuracy and Fairness of Decisions at the
Hearings Level, GAO-04-14 (Washington, D.C.: Nov. 12, 2003).
[28] See the Lewin Group, Inc., et al., Evaluation of SSA's Disability
Quality Assurance (QA) Processes and Development of QA Options That
Will Support the Long-Term Management of the Disability Program, (Falls
Church, VA: Mar. 16, 2001), pp. 168 and E-9.
[29] For additional information on SSA's progress and its risky
strategy for implementing its accelerated disability claims system, see
U.S. General Accounting Office, Electronic Disability Claims
Processing: Social Security Administration's Accelerated Strategy
Faces Significant Risks, GAO-03-984T (Washington, D.C.: July 24, 2003);
Social Security Administration: Subcommittee Questions Concerning
Efforts to Automate the Disability Claims Process, GAO-03-1113R
(Washington, D.C.: Sept. 5, 2003); and Electronic Disability Claims
Processing: SSA Needs to Address Risks Associated with Its Accelerated
Systems Development Strategy, GAO-04-466 (Washington, D.C.: Mar. 26,
2004).
[30] This excerpt is taken from a statement by the Honorable Jo Anne B.
Barnhart, Commissioner, Social Security Administration, Testimony
before the Subcommittee on Social Security of the House Committee on
Ways and Means, September 25, 2003.
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. General Accounting Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
