Questions for the Record Related to the Implementation of the Department of Defense's National Security Personnel System
Gao ID: GAO-09-669R May 18, 2009
This letter provides answers to questions for the record. Questions from Chairman Ortiz include: (1) What steps could have been taken to roll out NSPS in a more orderly and fair fashion and if NSPS continues, what steps should now be taken to move forward? (2) As DOD and OPM leadership hold discussions to determine the overall framework, scope, and timeline of the review, what guidance or suggestions would you give to DOD and OPM to include in the methodology of this study? (3) Under the GS system, an employee steadily moves up through the various grades and can actually monitor actual career progression. There appears to be no such similar movement in NSPS; an employee, while receiving pay increases and bonuses, may remain in the same pay band for his/her entire career. If this is a valid concern, how can it be addressed, if NSPS continues? Questions from Representative Forbes include: (1) Based on the GAO testimony, one safeguard GAO believes needs to be implemented to increase employee confidence in the pay for performance system is for DOD to have a third party analyze the pay-pool recommendations for "anomalies" before any final decision is made to determine whether an employee's rating accurately reflects the employee's performance and whether any non-merit based factors contributed to the "anomaly." Explain how GAO sees this third party analysis working. (2) In GAO's view, who would the third party be, a DOD entity or a non-DOD entity? (3) What criteria does GAO see as constituting an anomaly? (4) In investigating "blatant discrimination" or "egregious decisions" would the employee be contacted and interviewed? (5) Would a single third party be evaluating all 1,600 pay pools across DOD to get a DOD wide view of anomalies, or would 1,600 third-party reviews be conducted at each pay pool without regard for a comprehensive DOD look? (6) What effect would the third-party analyses have on the timeliness of the pay-pool process? (7) Would the pay pool decisions on all the other employees in the pay pool be held up until the "anomaly" was resolved? (8) How would GAO see the anomaly being corrected - a directive to the rater to change the rating, or some disciplinary action against the rater, or some other form of corrective action? (9) Does GAO see any appeal rights for the manager or employee involved in the "anomaly"?
Answers to Chairman Ortiz's questions: (1) One key need is to modernize performance management systems in executive agencies so that they are capable of adequately supporting more performance-based pay and other personnel decisions. DOD needs to assess and address employee engagement in the system. DOD has collected survey data and conducted focus groups of employees under NSPS, but it is missing a key piece--an action plan. (2) Moving forward, as DOD and OPM embark on a study of NSPS and review how NSPS operates and its underlying policies, DOD has a unique opportunity to consider our previous recommendations, as well as all of the other internal safeguards key to ensuring that performance management systems in the government are fair, effective, and credible. In addition to a review of internal safeguards, this study provides DOD the opportunity to look at employee engagement in the process and develop an action plan to address employee concerns about NSPS. (3) DOD needs to collect more information on what the issues are surrounding this employee perception on career progression, including the underlying causes and the extent of this concern, so that the department can determine if it is indeed a valid concern. Answers to Chairman Forbes' questions: (1) Given that each agency has its own set of unique challenges and its own approach for handling those challenges, we believe that the department is in the best position to determine how to appropriately design and implement a predecisional analysis for NSPS. (2) We would expect the third party to be a DOD entity that is removed from the chain of command--that is, the human capital office or an office of opportunity and inclusiveness. (3) An anomaly would be characterized as a set of ratings for which there is a statistically significant difference in comparison to the larger group. Please see GAO-09-669R for more information. (4) The predecisional reviews are to help achieve consistency in the performance management process and provide reasonable assurance that the performance decisions are merit-based and fair. Due to the nature of the investigation, the employee would not be contacted during an investigation. However, information provided by employees, such as the self-assessment, can be considered during the review process, as could information provided by responsible managers regarding underlying reasons for any anomalies. (5) Each agency has its own set of unique challenges and its own approach for handling those challenges. As a result, we feel that the department is in the best position to determine how to appropriately design and implement a predecisional analysis for NSPS. (6) It would likely add time to the existing process. However, we believe that it is important that DOD take steps to ensure that its employees' ratings are perceived as fair reflections of their performance. (7) Given that the predecisional review is intended to take place prior to the ratings being finalized but before they are certified and released to employees, all other ratings would not be released until the predecisional review was completed and appropriate responses (which could include inaction) were determined for any anomalies identified. (8) Where managers provide information that explains the merit-based factors and reasons for the anomalies, the managers would not change the ratings. On the other hand, managers could determine that some vital information was not considered that would provide a basis for changing the rating. (9) Although the third-party reviewer identifies the anomalies, it is the responsible manager that examines the basis underlying the ratings and is held accountable for ensuring the ratings are merit based. Please see GAO-09-669R for further information.
GAO-09-669R, Questions for the Record Related to the Implementation of the Department of Defense's National Security Personnel System
This is the accessible text file for GAO report number GAO-09-669R
entitled 'Questions for the Record Related to the Implementation of the
Department of Defense's National Security Personnel System' which was
released on May 18, 2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
GAO-09-669R:
United States Government Accountability Office:
Washington, DC 20548:
May 18, 2009:
The Honorable Solomon Ortiz:
Chairman:
Subcommittee on Readiness:
Committee on Armed Services:
House of Representatives:
Subject: Questions for the Record Related to the Implementation of the
Department of Defense's National Security Personnel System:
It was a pleasure to appear before your subcommittee on April 1, 2009,
to discuss the Department of Defense's (DOD) implementation of its new
human capital system for managing civilian personnel--the National
Security Personnel System (NSPS).[Footnote 1] This letter responds to
your request that I provide answers to questions for the record from
the hearing. The questions, along with my responses, follow.
Questions from Chairman Ortiz:
1. GAO noted that NSPS was implemented too quickly.
* What steps could have been taken to roll out NSPS in a more orderly
and fair fashion?
As we have previously reported, we support the need to expand broad
banding approaches and pay-for-performance-based systems in the federal
government.[Footnote 2] However, moving too quickly or prematurely to
implement such programs, whether at DOD or elsewhere, can significantly
raise the risk of doing it incorrectly. Hasty implementation could also
set back the legitimate need to move to a more performance-and results-
based system for the federal government as a whole. Thus, while it is
imperative that we take steps to better link employee pay to
performance across the federal government, how it is done, when it is
done, and the basis on which it is done can make all the difference in
whether or not such efforts are successful. In our view, one key need
is to modernize performance management systems in executive agencies so
that they are capable of adequately supporting more performance-based
pay and other personnel decisions.
While our previous work does not prescribe a process and time frames
for rolling out systems such as NSPS, we have stressed that agencies
should have an institutional infrastructure in place that would
include, at a minimum, (1) a human capital planning process that
integrates the agency's human capital policies, strategies, and
programs with its program goals and mission and desired outcomes; (2)
the capabilities to effectively develop and implement a new human
capital system; and (3) the existence of a modern, effective, and
credible performance management system that includes adequate
safeguards, including reasonable transparency and appropriate
accountability mechanisms, to ensure the fair, effective, and
nondiscriminatory implementation of a new system. Prior to NSPS
implementation, we cautioned that, while the DOD leadership had the
intent and the ability to implement the needed infrastructure, it did
not have the necessary infrastructure in place across the department.
Further, our work has continued to stress the importance of
incorporating internal safeguards into the design and implementation of
large-scale pay-for-performance programs. In 2008, we evaluated DOD's
efforts to implement nine safeguards and accountability mechanisms.
[Footnote 3] We found that, while DOD had taken some steps to implement
internal safeguards to ensure that NSPS is fair, effective, and
credible, the implementation of some safeguards could be improved.
First, DOD does not require a third party to analyze rating results for
anomalies prior to finalizing employee ratings, and therefore it is
unable to ensure that ratings are fair and nondiscriminatory before
they are finalized. Second, the process has lacked transparency until
recently because DOD did not require commands to publish final rating
distributions, though doing so was recognized as a best practice by
NSPS program officials at all four components. In 2008, the department
revised its NSPS regulations and guidance to require commands to
publish the final overall rating results. Third, NSPS guidance may
discourage rating officials from making meaningful distinctions in
employee ratings because it indicated that the majority of employees
should be rated at the "3" level, on a scale of 1 to 5, resulting in a
hesitancy to award ratings in other categories. We continue to believe
that improved implementation of these safeguards will help bolster
employee confidence in the system and ensure that the system is fair,
effective, and credible.
* If NSPS continues, what steps should now be taken to move forward?
We have previously reported that converting to NSPS was a significant
transition for the department.[Footnote 4] We have further reported
that it will take time for employees to accept the system, based on the
studies conducted by the Office of Personnel Management (OPM) on the
federal government demonstration projects for performance management.
First, and foremost, DOD needs to assess and address employee
engagement in the system. DOD has collected survey data and conducted
focus groups of employees under NSPS, but it is missing a key piece--an
action plan. Our 2008 report recommended that DOD develop and implement
a specific action plan to address employee perceptions of NSPS
ascertained from feedback avenues such as, but not limited to, DOD's
survey and DOD's and GAO's employee focus groups. At a minimum, this
plan should include actions to mitigate employee concerns about the
potential influence that employees' and supervisors' writing skills
have on the panels' assessment of employee ratings and the lack of
transparency and understanding of the pay pool panel process. Such a
plan would demonstrate to employees that the department is listening to
their concerns and making plans to address, as appropriate, the
concerns that are identified. In short, DOD needs to tell the employees
that they are going to take action on their concerns. In addition, the
recently announced study by DOD and OPM is an opportunity to assess the
status of the system. While the review intends to include a thorough
examination of all NSPS policies, regulations, and practices, we would
like to see DOD leverage this opportunity to assess for itself how the
department is implementing internal safeguards. Specifically, we are
interested in an update of how the safeguards have been incorporated
into their policies and how the safeguards are working.
2. The Deputy Secretary of Defense, in cooperation with the Office of
Personnel Management recently announced that the department would halt
conversions of DOD civilian employees to NSPS, pending the outcome of a
review by DOD and OPM. The proposed review will assess whether or not
NSPS is meeting its objectives of being a fair, transparent, and
effective personnel system. Finalizing the details of such a review's
overall framework, scope, timeline, and leadership will take time.
* As DOD and OPM leadership hold discussions to determine the overall
framework, scope, and timeline of the review, what guidance or
suggestions would you give to DOD and OPM to include in the methodology
of this study?
As we have previously reported, the extent to which DOD incorporates
internal safeguards into the design and implementation of NSPS and how
it addresses employee perceptions of NSPS are key to the success of the
system.[Footnote 5] Moving forward, as DOD and OPM embark on a study of
NSPS and review how NSPS operates and its underlying policies, DOD has
a unique opportunity to consider our previous recommendations, as well
as all of the other internal safeguards key to ensuring that
performance management systems in the government are fair, effective,
and credible. In addition to a review of internal safeguards, this
study provides DOD the opportunity to look at employee engagement in
the process and develop an action plan to address employee concerns
about NSPS. As we approached our work, we used a methodology that
systematically took into account employee input from all levels. We
used a combination of survey analysis, interviews, and employee
discussion groups to obtain information on employee perceptions. In
general, the combination of employee surveys with interviews or
discussion groups is helpful because it yields useful information at
the population level, as well as the individual employee experience
level.
3. One concern expressed by employees who have converted from GS to
NSPS is that there is no real career progression. Under the GS system,
an employee steadily moves up through the various grades and can
actually monitor actual career progression. There appears to be no such
similar movement in NSPS; an employee, while receiving pay increases
and bonuses, may remain in the same pay band for his/her entire career.
* If this is a valid concern, how can it be addressed, if NSPS
continues?
First, DOD needs to collect more information on what the issues are
surrounding this employee perception on career progression, including
the underlying causes and the extent of this concern, so that the
department can determine if it is indeed a valid concern. For example,
is there an issue with lack of career progression or are employees
perceiving that there is an issue as a result of lack of communication
or education on the new system? In our 2008 report, we recommended that
the department develop and implement a specific action plan to address
employee perceptions of NSPS ascertained from feedback avenues such as,
but not limited to, DOD's survey and DOD's and GAO's employee focus
groups. We believe that this is another example of how the department
could use such an action plan to guide its approach for addressing
employee concerns. Specifically, the plan may incorporate various
communication and education strategies to help employees understand how
the shift from pay grades to pay bands still affords them opportunities
for professional development, as well as movement through the pay band.
While we acknowledge that change takes time to gain employee acceptance
and that the implementation of NSPS is a large-scale organizational
transformation, employee concerns, such as these, must be heard and
addressed accordingly in order to ensure greater employee acceptance
and, ultimately, successful implementation of the NSPS performance
management system.
Questions from Representative Forbes:
Based on the GAO testimony, one safeguard GAO believes needs to be
implemented to increase employee confidence in the pay for performance
system is for DOD to have a third party analyze the pay-pool
recommendations for "anomalies" before any final decision is made to
determine whether an employee's rating accurately reflects the
employee's performance and whether any non-merit based factors
contributed to the "anomaly."
1. Explain how you see this third party analysis working.
Given that each agency has its own set of unique challenges and its own
approach for handling those challenges, we believe that the department
is in the best position to determine how to appropriately design and
implement a predecisional analysis for NSPS. That said, we believe that
the third-party analysis should be conducted by an independent
reviewing office, such as a human capital office, that is able to
conduct the analysis outside of the chain of command. Taking the
analysis outside of the chain of command helps to ensure that the
process remains as independent as possible. Seeing that DOD currently
has over 200,000 civilian employees under NSPS, the department could
consider phasing in the third-party analysis by starting with a
representative sample of employees. A phased implementation approach
recognizes that different components of agencies will often have
different levels of readiness and different capabilities to implement
new authorities. Moreover, a phased approach allows for learning so
that appropriate adjustments and midcourse corrections can be made
before new policies and procedures are fully implemented
organizationwide.
2. In your view, who would the third party be, a DOD entity or a non-
DOD entity?
As noted in our response to question 1, we would expect the third party
to be a DOD entity that is removed from the chain of command--that is,
the human capital office or an office of opportunity and inclusiveness.
3. What criteria does GAO see as constituting an anomaly?
Generally, an anomaly would be characterized as a set of ratings for
which there is a statistically significant difference in comparison to
the larger group. For example, if the data indicated that there was a
statistically significant difference between the ratings of a
particular subset of the larger workforce compared to the larger group
at that same level, this could constitute an anomaly. The presence of
an anomaly is not alone proof that there is a problem. Rather,
identifying an anomaly in the data prior to finalizing the rating
decisions would enable management to investigate the situation and
determine whether the results are justified and merit-based.
4. In investigating "blatant discrimination" or "egregious decisions"
would the employee be contacted and interviewed?
The predecisional reviews are to help achieve consistency in the
performance management process and provide reasonable assurance that
the performance decisions are merit-based and fair. Due to the nature
of the investigation, the employee would not be contacted during an
investigation. However, information provided by employees, such as the
self-assessment, can be considered during the review process, as could
information provided by responsible managers regarding underlying
reasons for any anomalies.
5. Would a single third party be evaluating all 1,600 pay pools across
DOD to get a DOD wide view of anomalies, or would 1,600 third-party
reviews be conducted at each pay pool without regard for a
comprehensive DOD look?
As noted in our response to question 1, each agency has its own set of
unique challenges and its own approach for handling those challenges.
As a result, we feel that the department is in the best position to
determine how to appropriately design and implement a predecisional
analysis for NSPS. One approach, as noted in our response to question
1, would be for DOD to phase in the third party analysis by starting
with a representative sample of employees. Such an approach recognizes
that different levels of readiness and different capabilities exist
among agency components and allows for learning so that appropriate
adjustments and midcourse corrections can be made before full
implementation.
6. What effect would the third-party analyses have on the timeliness of
the pay-pool process?
It would likely add time to the existing process. However, we believe
that it is important that DOD take steps to ensure that its employees'
ratings are perceived as fair reflections of their performance. Taking
additional time to complete a predecisional analysis is one safeguard
that DOD can implement to raise employee confidence in the fairness and
credibility of the system.
7. Would the pay pool decisions on all the other employees in the pay
pool be held up until the "anomaly" was resolved?
Given that the predecisional review is intended to take place prior to
the ratings being finalized but before they are certified and released
to employees, all other ratings would not be released until the
predecisional review was completed and appropriate responses (which
could include inaction) were determined for any anomalies identified.
8. How would GAO see the anomaly being corrected - a directive to the
rater to change the rating, or some disciplinary action against the
rater, or some other form of corrective action?
Where managers provide information that explains the merit-based
factors and reasons for the anomalies, the managers would not change
the ratings. On the other hand, managers could determine that some
vital information was not considered that would provide a basis for
changing the rating. In all cases, it is the unit manager, not the
third party conducting the predecisional review, that would determine
whether a change would be warranted. Further, the review is not
intended to change the results to portray an "ideal" distribution, or
to alter the outcome of the performance management process. The purpose
of the predecisional review is to identify if anomalies exist and, if
found, inform managers of the need for further review to provide
reasonable assurance that the basis for each rating is fair, credible,
and merit-based.
9. Does GAO see any appeal rights for the manager or employee involved
in the "anomaly"?
Although the third-party reviewer identifies the anomalies, it is the
responsible manager that examines the basis underlying the ratings and
is held accountable for ensuring the ratings are merit based. While it
is unlikely, should a manager be disciplined, he or she might have
appeal rights, depending on the nature of the discipline imposed.
Any employee has a right to appeal his or her final rating. However,
these appeal rights do not apply to the predecisional review process
since it is intended to take place prior to the ratings being finalized
but before they are certified and released to employees.
For additional information on our work on human capital issues at DOD,
please contact me on 202-512-3604 or farrellb@gao.gov.
Signed by:
Brenda S. Farrell:
Director:
Defense Capabilities and Management:
[End of section]
Footnotes:
[1] GAO, Human Capital: Improved Implementation of Safeguards and an
Action Plan to Address Employee Concerns Could Increase Employee
Acceptance of the National Security Personnel System, [hyperlink,
http://www.gao.gov/products/GAO-09-464T] (Washington, D.C.: April 1,
2009).
[2] GAO, Defense Transformation: DOD's Proposed Civilian Personnel
System and Governmentwide Human Capital Reform, [hyperlink,
http://www.gao.gov/products/GAO-03-741T] (Washington, D.C.: May 1,
2003).
[3] GAO, Human Capital: DOD Needs to Improve Implementation of and
Address Employee Concerns about Its National Security Personnel System,
[hyperlink, http://www.gao.gov/products/GAO-08-773] (Washington, D.C.:
Sept. 10, 2008).
[4] [hyperlink, http://www.gao.gov/products/GAO-08-773].
[5] [hyperlink, http://www.gao.gov/products/GAO-08-773].
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
