Elections
DOD Can Strengthen Evaluation of Its Absentee Voting Assistance Program Gao ID: GAO-10-476 June 17, 2010Under the Uniformed and Overseas Citizens Absentee Voting Act (UOCAVA), citizens covered are permitted to register and vote absentee. The Secretary of Defense has the primary responsibility for federal UOCAVA functions, and the Department of Defense's (DOD) Federal Voting Assistance Program (FVAP) facilitates absentee voters' participation in federal elections. Since 2001, the DOD Office of Inspector General and GAO have reviewed FVAP's efforts and recommended improvements to its procedures and the direction FVAP provides to the services. In response to a congressional request, this report evaluates how FVAP (1) addresses its mission and evaluates efforts to conduct it, (2) aligns budget priorities with strategic goals, and (3) implemented DOD Office of Inspector General and GAO recommendations made from 2001 through 2009. GAO analyzed FVAP's performance measures, relevant DOD directives, FVAP's strategic plans, budgets, and past audit reports. Also, GAO interviewed agency officials.
FVAP has efforts under way to addressrequirements identified in DOD guidance, but FVAP's evaluation of those efforts yields data of varying quality. FVAP's efforts to address requirements include obtaining and distributing state-specific voting information and developing forms to request absentee voting materials. To assess the effectiveness of FVAP's efforts, GAO used criteria (e.g., data credibility, reliability, and consistency) that it had previously identified. While FVAP improved some of its evaluative methodologies during this decade, GAO identified concerns with findings for FVAP's postelection surveys and "measures of success." The concerns include low response rates and not following governmentwide guidance to conduct nonresponse analyses as well as credible but limited measures to assess some efforts. Also, FVAP has not evaluated its Voting Assistance Officer training even though Voting Assistance Officers are crucial to FVAP's voter outreach efforts. GAO noted the need for training programs to have an evaluative component. FVAP's new Director commented on reasons for GAO findings, explaining that many organizations focus on implementation instead of impact, but he plans to add more evaluations in the future. Better evaluative information on its efforts and an assessment of its Voting Assistance Officer training would allow FVAP to enhance its efforts to be effective and efficient. While a detailed analysis was not possible because FVAP does not budget by strategic goals, FVAP's recent budgets and current strategic goals appear to be generally aligned. GAO's review found linkages between FVAP's recent annual budgets that have averaged about $4 million and its strategic goals. The strategic plan for 2008 and 2009 contained four general goals, including encouraging adoption of FVAP legislative initiatives in order to facilitate UOCAVA voting and improving marketing and outreach efforts. The linkages to the goals were most readily apparent for the budget categories of travel and contracts. For example, FVAP's travel budget was higher during years with federal elections, in large part due to travel to conduct Voting Assistance Officer training workshops and in support of FVAP's objective to enhance training products and services. In addition, FVAP's contracts budget for products and services, such as improving the Web site and exploring new technology methods for Internet voting, are linked to goals involving the improvement of the FVAP's marketing and outreach efforts with stakeholders. FVAP and the Under Secretary of Defense for Personnel and Readiness fully or partially implemented most UOCAVA-related audit recommendations from 2001 through 2009. FVAP and the Under Secretary fully or partially implemented 11 of 16 DOD Office of Inspector General and GAO recommendations for improvements in the oversight and direction it provides to the services. These included several changes to DOD guidance to specify the number of Voting Assistance Officers needed and emphasize the services' responsibilities in monitoring and overseeing their programs. Also, FVAP fully or partially implemented six of the seven recommendations for improvements in its processes, such as improving the security of its electronic initiatives. GAO recommends that DOD (1) assess its current evaluation methodologies, (2) implement additional methodologies, and (3) evaluate methods for training voting assistance officers. DOD concurred or partially concurred with GAO's recommendations.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Jack E. Edwards Team: Government Accountability Office: Defense Capabilities and Management Phone: (202) 512-8246GAO-10-476, Elections: DOD Can Strengthen Evaluation of Its Absentee Voting Assistance Program
This is the accessible text file for GAO report number GAO-10-476
entitled 'Elections: DOD Can Strengthen Evaluation of Its Absentee
Voting Assistance Program' which was released on July 21, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
June 2010:
Elections:
DOD Can Strengthen Evaluation of Its Absentee Voting Assistance
Program:
GAO-10-476:
GAO Highlights:
Highlights of GAO-10-476, a report to congressional requesters.
Why GAO Did This Study:
Under the Uniformed and Overseas Citizens Absentee Voting Act
(UOCAVA), citizens covered are permitted to register and vote
absentee. The Secretary of Defense has the primary responsibility for
federal UOCAVA functions, and the Department of Defense‘s (DOD)
Federal Voting Assistance Program (FVAP) facilitates absentee voters‘
participation in federal elections. Since 2001, the DOD Office of
Inspector General and GAO have reviewed FVAP‘s efforts and recommended
improvements to its procedures and the direction FVAP provides to the
services. In response to a congressional request, this report
evaluates how FVAP (1) addresses its mission and evaluates efforts to
conduct it, (2) aligns budget priorities with strategic goals, and (3)
implemented DOD Office of Inspector General and GAO recommendations
made from 2001 through 2009. GAO analyzed FVAP‘s performance measures,
relevant DOD directives, FVAP‘s strategic plans, budgets, and past
audit reports. Also, GAO interviewed agency officials.
What GAO Found:
FVAP has efforts under way to address requirements identified in DOD
guidance, but FVAP‘s evaluation of those efforts yields data of
varying quality. FVAP‘s efforts to address requirements include
obtaining and distributing state-specific voting information and
developing forms to request absentee voting materials. To assess the
effectiveness of FVAP‘s efforts, GAO used criteria (e.g., data
credibility, reliability, and consistency) that it had previously
identified. While FVAP improved some of its evaluative methodologies
during this decade, GAO identified concerns with findings for FVAP‘s
postelection surveys and ’measures of success.“ The concerns include
low response rates and not following governmentwide guidance to
conduct nonresponse analyses as well as credible but limited measures
to assess some efforts. Also, FVAP has not evaluated its Voting
Assistance Officer training even though Voting Assistance Officers are
crucial to FVAP‘s voter outreach efforts. GAO noted the need for
training programs to have an evaluative component. FVAP‘s new Director
commented on reasons for GAO findings, explaining that many
organizations focus on implementation instead of impact, but he plans
to add more evaluations in the future. Better evaluative information
on its efforts and an assessment of its Voting Assistance Officer
training would allow FVAP to enhance its efforts to be effective and
efficient.
While a detailed analysis was not possible because FVAP does not
budget by strategic goals, FVAP‘s recent budgets and current strategic
goals appear to be generally aligned. GAO‘s review found linkages
between FVAP‘s recent annual budgets that have averaged about $4
million and its strategic goals. The strategic plan for 2008 and 2009
contained four general goals, including encouraging adoption of FVAP
legislative initiatives in order to facilitate UOCAVA voting and
improving marketing and outreach efforts. The linkages to the goals
were most readily apparent for the budget categories of travel and
contracts. For example, FVAP‘s travel budget was higher during years
with federal elections, in large part due to travel to conduct Voting
Assistance Officer training workshops and in support of FVAP‘s
objective to enhance training products and services. In addition, FVAP‘
s contracts budget for products and services, such as improving the
Web site and exploring new technology methods for Internet voting, are
linked to goals involving the improvement of the FVAP‘s marketing and
outreach efforts with stakeholders.
FVAP and the Under Secretary of Defense for Personnel and Readiness
fully or partially implemented most UOCAVA-related audit
recommendations from 2001 through 2009. FVAP and the Under Secretary
fully or partially implemented 11 of 16 DOD Office of Inspector
General and GAO recommendations for improvements in the oversight and
direction it provides to the services. These included several changes
to DOD guidance to specify the number of Voting Assistance Officers
needed and emphasize the services‘ responsibilities in monitoring and
overseeing their programs. Also, FVAP fully or partially implemented
six of the seven recommendations for improvements in its processes,
such as improving the security of its electronic initiatives.
What GAO Recommends:
GAO recommends that DOD (1) assess its current evaluation
methodologies, (2)implement additional methodologies, and (3) evaluate
methods for training voting assistance officers. DOD concurred or
partially concurred with GAO‘s recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-10-476] or key
components. For more information, contact Jack Edwards (202) 512-8246
or edwardsj@gao.gov.
[End of section]
Contents:
Letter:
Background:
FVAP Has Taken Steps to Address DOD Requirements, but the Quality of
the Evaluative Information Varies:
FVAP's Budget Priorities Appear to Be Generally Aligned with Its
Strategic Plan Goals:
FVAP and USD (P&R) Fully or Partially Implemented Most of the
Recommendations:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope & Methodology:
Appendix II: DOD Office of Inspector General and GAO Reports
Containing DOD Voting Assistance Program Recommendations:
Appendix III: Comments from the Department of Defense:
Appendix IV: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: FVAP's Efforts to Address the Requirements in DOD Directive
1000.04:
Table 2: FVAP's Budgets for Fiscal Years 2005 through 2009 Broken into
Six Major Categories:
Table 3: FVAP's and USD (P&R)'s Implementation of DOD Office of
Inspector General and GAO Recommendations: 2001 through 2009:
Table 4: DOD Office of Inspector General and GAO Recommendations to
Improve Oversight of and Direction to the Services and FVAP's and USD
(P&R)'s Actions Taken to Implement the Recommendations: 2001 through
2009:
Table 5: DOD Office of Inspector General and GAO Recommendations to
Improve FVAP Processes and Actions Taken by FVAP to Implement the
Recommendations: 2001 through 2009:
Abbreviations:
DOD: Department of Defense:
FVAP: Federal Voting Assistance Program:
UOCAVA: Uniformed and Overseas Citizens Absentee Voting Act:
USD (P&R): Under Secretary of Defense for Personnel and Readiness:
VAO: Voting Assistance Officer:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
June 17, 2010:
Congressional Requesters:
Members of the military, their dependents of voting age, and other
eligible U.S. citizens living abroad are permitted to participate by
absentee ballot in all federal elections under the Uniformed and
Overseas Citizens Absentee Voting Act (UOCAVA).[Footnote 1] According
to the Department of Defense (DOD), the act covers over 6 million U.S.
citizens. The Secretary of Defense is the presidential designee with
the primary responsibility for federal UOCAVA functions; and through
the Under Secretary of Defense for Personnel and Readiness (USD
(P&R)), the Federal Voting Assistance Program (FVAP) provides support
to potential overseas absentee voters in order to facilitate their
participation in federal elections. Absentee voters may, however,
experience difficulties accessing voting materials and then returning
them in time to meet state-specific deadlines.
According to FVAP's 2008-2009 strategic plan, the program's mission is
to inform and educate U.S. citizens worldwide about their right to
vote; foster voter participation; and protect the integrity of and
enhance the electoral process at the federal, state, and local levels.
Among other things, FVAP staff train voting assistance officers (VAOs)
from the services, the Department of State, and overseas citizen
organizations to help these entities implement their voting assistance
programs. Since 2001, the DOD Office of Inspector General and GAO have
reviewed FVAP's efforts and made recommendations to USD (P&R) to
improve the oversight and direction FVAP provides to the services and
to improve FVAP's processes.
Following the 2008 presidential election, you requested that we assess
FVAP's efforts to provide assistance to absentee voters. In response
to your request, we evaluated the extent to which FVAP has (1)
addressed its mission and evaluated the effectiveness of its efforts
to conduct that mission, (2) put processes in place to help ensure
that budget priorities are aligned with the strategic plans and goals,
and (3) implemented recommendations made from 2001 through 2009 by the
DOD Office of Inspector General and GAO. This report contributes to a
larger GAO body of work on DOD's overseas absentee voting program (see
the Related GAO Products section at the end of this report).
To evaluate the extent to which FVAP has addressed its mission and
evaluated the effectiveness of its efforts to conduct that mission, we
analyzed FVAP's documentation of its efforts, such as communications
with VAOs and state election officials, and the relationship of those
efforts to FVAP's mission requirements outlined in UOCAVA and related
DOD guidance. We corroborated our findings during interviews with FVAP
officials. To assess the performance measures that FVAP used to
evaluate the extent to which its efforts addressed its mission, we
reviewed FVAP's data gathering procedures and available measures
obtained from FVAP and published reports (e.g., ours and those of the
DOD Office of Inspector General and non-governmental organizations).
We then compared FVAP's procedures and measures against criteria on
performance measurement that GAO and others had published in previous
reports.[Footnote 2] For example, we reviewed the methodology and
questions used in FVAP's 2008 postelection surveys of servicemembers,
VAOs, overseas citizens, and local election officials and compared
them against survey-related criteria in guidance from the Office of
Management and Budget.[Footnote 3] To evaluate the extent to which
FVAP has processes in place to align its budget priorities with its
strategic plans and goals, we reviewed FVAP's budget processes and
compared its budgets for fiscal years 2005 through 2009 with the goals
described in its strategic plan for 2008 through 2009. To determine
the extent to which the DOD Office of Inspector General's and GAO's
recommendations were implemented, we reviewed DOD's Office of
Inspector General and GAO's reports issued from 2001 through 2009. We
also interviewed the DOD Office of Inspector General officials who had
conducted the evaluations of FVAP's and USD (P&R)'s actions to
implement recommendations and reviewed DOD directives, a DOD draft
instruction, USD (P&R) memorandums, and DOD Office of Inspector
General follow-up records of these evaluations to help determine the
extent to which FVAP and USD (P&R) had implemented the DOD Office of
Inspector General's recommendations. Similarly, we reviewed our
reports and follow-up records to determine the extent to which GAO
recommendations had been implemented. For each of our objectives, we
assessed the reliability of the data we analyzed by reviewing existing
documentation related to the data sources and interviewing
knowledgeable agency officials about the data that we used. We found
the data sufficiently reliable for the purposes of this report.
Appendix I explains our scope and methodology in greater detail.
We conducted this performance audit from June 2009 through May 2010 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Background:
FVAP relies heavily on the military services and the Department of
State to carry out mission requirements to help ensure that U.S.
citizens covered by UOCAVA understand their voting rights and how to
register and vote by absentee ballot. Each military service is also
required to designate a senior service voting representative to manage
the service's voting assistance program. Similarly, the Department of
State--through its Bureau of Consular Affairs, embassies, and
consulates--carries out a portion of its voter assistance
responsibilities through staff designated to provide assistance.
To accomplish its mission, FVAP also coordinates and interacts with a
number of other stakeholders including the Election Assistance
Commission (Commission), state and local election officials, and non-
governmental organizations. For example, the Commission was directed
to establish electronic absentee-voting system guidelines for DOD and
FVAP to use in establishing an electronic absentee-voting
demonstration project as directed by section 1604 of the National
Defense Authorization Act for Fiscal Year 2002.[Footnote 4] FVAP also
works with state and local election officials to promote, among other
things, the adoption of legislative initiatives to facilitate absentee
voting under UOCAVA and to help ensure mutual understanding of state-
specific absentee-voting procedures. Finally, FVAP coordinates with
advocacy groups and other non-governmental organizations like the
Overseas Vote Foundation and the Pew Charitable Trusts to leverage the
outreach capacity of those organizations for reaching the diffuse
population of overseas U.S. citizens.
Findings from our past reports have documented opportunities for
improving FVAP efforts to meet mission requirements. For example, in
2001, we reported on opportunities for DOD to improve outreach to
absentee voters and improve the oversight of FVAP. In 2007, we
reported on opportunities for DOD to improve electronic absentee
voting initiatives--one opportunity's being for DOD to develop action
plans to identify safeguards for the security and privacy of DOD's
electronic and Internet-accessible voting systems.
FVAP Has Taken Steps to Address DOD Requirements, but the Quality of
the Evaluative Information Varies:
FVAP Has Taken Steps to Address Requirements in the DOD Directive That
Implements UOCAVA:
FVAP has efforts under way to address requirements set out in the DOD
guidance implementing the executive order assigning primary
responsibility for federal UOCAVA functions to the Secretary of
Defense.[Footnote 5] DOD Directive 1000.04 (DOD's guidance) specifies
that USD (P&R) is responsible for managing, coordinating, and
performing the responsibilities assigned to the presidential designee
under UOCAVA, and for administering FVAP.[Footnote 6] The directive
also requires USD (P&R) to designate a Director for FVAP who is to be
responsible for all aspects of the program, and the directive contains
a number of requirements that address the responsibilities assigned in
UOCAVA. Table 1 describes requirements in DOD's Directive 1000.04 and
identifies FVAP's efforts to address them. For example, to address the
requirement to publicize the right of citizens to participate in the
electoral process under UOCAVA, FVAP prints and issues voter
literature (e.g., posters) and has downloadable versions of literature
and other information on FVAP's Web site. Among other things, these
materials provide the absentee voter with information on how to
contact FVAP's ombudsman service, as well as methods and a timeline
for both requesting an application for absentee voting and returning
an absentee ballot.
Table 1: FVAP's Efforts to Address the Requirements in DOD Directive
1000.04:
DOD-specified requirements: Prescribe an official postcard form,
containing both an absentee voter registration and absentee ballot
registration application for use by the states, as part of the UOCAVA
responsibilities referenced in the directive;
FVAP's efforts to address requirements: Developed Federal Post Card
Application (Standard Form-76A) that all states accept as an
application for the absentee voter registration and ballot.
DOD-specified requirements: Prescribe a federal write-in absentee
ballot for use in general elections for federal office by absent
uniformed services voters and overseas voters who make timely
application for, and do not receive, states' absentee ballots as part
of the UOCAVA responsibilities referenced in the directive;
FVAP's efforts to address requirements: Developed Federal Write-In
Absentee Ballot (Standard Form-186A) that all states accept for
absentee voting in federal elections.
DOD-specified requirements: Prescribe the standard oath to be used
with any UOCAVA documents[A];
FVAP's efforts to address requirements: Prepared the standard oath and
prints it on official absentee voting application and ballot.
DOD-specified requirements: Designate a biennial Armed Forces Voters
Week to encourage military personnel and their family members to
exercise their right to vote;
FVAP's efforts to address requirements: Identifies and communicates
the dates of the Armed Forces Voters Week to the services.
DOD-specified requirements: Obtain current voting information from
each state, and disseminate it to other federal executive departments,
agencies, and DOD components;
FVAP's efforts to address requirements: Updates deadlines and other
state-specific requirements in the Voting Assistance Guide, provides
it to VAOs, and posts the guide on the FVAP Web site.
DOD-specified requirements: Publicize the right of citizens to
participate in the electoral process under UOCAVA;
FVAP's efforts to address requirements: Issues voter literature (e.g.,
posters), posts downloadable literature and other information on the
FVAP Web site, and issues press releases that publicize information on
UOCAVA-covered citizens' right to vote.
DOD-specified requirements: Provide an ombudsman service for all
UOCAVA-covered persons and state and local officials;
FVAP's efforts to address requirements: Provides a toll-free telephone
ombudsman service to assist VAOs, absentee voters, and state and local
officials with their questions and paperwork.
DOD-specified requirements: Ensure voters receive information about
registration, voting procedures, and materials pertaining to upcoming
elections;
FVAP's efforts to address requirements: Distributes Voting Assistance
Guides, Federal Post Card Applications, and other materials to
absentee voters and VAOs; and posts these materials on the FVAP Web
site.
DOD-specified requirements: Establish and maintain liaison with
officials of state legislatures and with state and local government
officials;
FVAP's efforts to address requirements: Maintains contact with state
and local officials, including issuing legislative initiative letters
and newsletters to identify how a state can meet requirements and
incorporate technology into the absentee voting process.
DOD-specified requirements: Encourage and assist states and other U.S.
jurisdictions to adopt the mandatory and recommended UOCAVA
provisions, and advise them on how federal laws and regulations apply
to their individual electoral systems;
FVAP's efforts to address requirements: Provides state-specific
legislative initiative letters detailing UOCAVA requirements and
recommendations to accommodate technological advances in absentee
voting.
DOD-specified requirements: Conduct voting assistance workshops for
VAOs in even-numbered years worldwide;
FVAP's efforts to address requirements: Provides in-person and online
training workshops to train VAOs on how to help potential absentee
voters understand their voting rights.
DOD-specified requirements: Survey military and other U.S. citizens
covered by UOCAVA to gather information for the quadrennial report to
the President and Congress required by UOCAVA;
FVAP's efforts to address requirements: Conducts postelection surveys
of servicemembers, VAOs, overseas citizens, and local election
officials; and issues quadrennial report to the President and Congress.
Source: GAO analysis of DOD data.
[A] The standard oath requires voters to "affirm that a material
misstatement of fact in the completion of such a document may
constitute grounds for conviction for perjury."
[End of table]
FVAP's Evaluations of Its Efforts to Address the Requirements Yield
Data of Varying Quality:
The evaluative information on FVAP's efforts varies in terms of
quality. In a prior report, we noted that agencies need reliable
information on the performance of agency programs, the financial
condition of programs and their operations, and the costs of programs
and operations in order to efficiently and effectively operate,
manage, and oversee activities.[Footnote 7] We also have identified
the reliability, consistency, and credibility of data as some of the
characteristics that might be examined in assessing the quality of
information.[Footnote 8]
During the last decade, we and the DOD Office of Inspector General
have identified opportunities for FVAP to enhance the evaluative
information that the agency has for monitoring and adjusting its
efforts, and FVAP took steps to improve its evaluative information.
For example, in 2001, we recommended that FVAP develop a methodology
to gather nationally projectable data on disqualified military and
overseas absentee ballots and reasons for disqualification. In
response, FVAP redesigned its postelection survey of local election
officials to obtain the number of absentee ballots not counted and the
reasons for the disqualification of absentee ballots from
servicemembers and other overseas citizens. Similarly, as part of its
evaluation of the services' voting assistance programs for 2007, the
DOD Office of Inspector General found that the services were not using
consistent metrics and targets when reporting on the effectiveness and
compliance of their programs. In response to this concern, FVAP
established with the services some performance metrics that were
published in the DOD Office of Inspector General's subsequent annual
assessment of the services' voting assistance programs.[Footnote 9]
FVAP refers to these new performance measures as its "measures of
success."
During our discussions with FVAP officials about how the agency
evaluates its efforts to address the requirements in the DOD
directive, the officials most frequently cited findings from FVAP's
postelection surveys and measures of success. As a result of our
review of documents and discussions with DOD staff knowledgeable about
the methodologies, we identified concerns regarding some of the
resulting evaluative information.
* Postelection surveys. Following each presidential election, FVAP
administers variations of this survey to six categories of individuals
covered or impacted by UOCAVA: uniformed servicemembers, federal
civilians overseas, non-federal civilians overseas, unit VAOs, State
Department VAOs, and local election officials. Respondents complete a
survey and submit the form directly to FVAP, which then compiles the
results. As part of recent improvements to its survey process, FVAP
collaborated with survey professionals in another part of DOD to,
among other things, re-word some survey questions and adjust sampling
methods in an effort to improve the statistical validity and
generalizability of the survey results. Despite these improvements,
concerns remain regarding the evaluative information resulting from
the surveys. For example, even though the response rates for some of
the surveys are low, FVAP does not conduct a non-response bias
analysis that the guidance from the Office of Management and Budget
has identified as a necessary step in determining whether survey
findings are biased.[Footnote 10] Not conducting such an analysis
limits data reliability, a data collection characteristic that GAO
previously identified as affecting information quality. We also talked
about credibility of data, another GAO-identified characteristic that
might be examined in assessing the quality of the survey findings. We
noted that potentially important content information was not included
as survey questions. For example, FVAP's survey of election officials
does not ask whether the state-specific legislative initiative letters
that FVAP sends were useful or how FVAP may assist states in better
meeting the provisions of UOCAVA. Without incorporating a wider
selection of credible, valid questions into its surveys, FVAP cannot
be assured that its efforts are effectively and efficiently addressing
stakeholders' concerns. Together, not conducting the additional
analyses to improve data reliability and leaving out some content in
its surveys may result in FVAP obtaining incomplete evaluative
information from its postelection surveys.
* Measures of success. The measures of success include the number of
VAOs trained, number of Federal Post Card Applications and Voting
Assistance Guides distributed, and number of times voting messages
were included on leave and earning statements, among other measures.
Such measures are one type of information that FVAP uses to assess two
requirements identified in the DOD directive: (1) ensure voters
receive information about registration, voting procedures, and
materials pertaining to upcoming elections and (2) conduct voting
assistance workshops for VAOs in even-numbered years worldwide. While
the measures of success are credible metrics for evaluating some of
FVAP's efforts, FVAP did not conduct a reliability assessment of the
data for the measures before the DOD Office of Inspector General used
the information in the 2009 report. In fact, we found that some of the
measures may be artificially high or low, indicating reliability
concerns. For example, because FVAP uses multiple methods to deliver
the Federal Post Card Application for absentee ballots, asking VAOs to
report the number of copies distributed could result in (1) over-
counting of servicemembers if they received multiple copies of the
post card through multiple outreach efforts or (2) under-counting of
servicemembers if some servicemembers only obtained a copy of the post
card from FVAP's Web site. We identified additional inconsistencies in
the FVAP-reported data, such as one service's total number of trained
VAOs was reported as approximately 2,500 persons greater than its
total number of VAOs. Also, even though the data are self-reported,
FVAP did not instruct either the VAOs or the services how to collect
the evaluative information in order to promote data consistency. In
addition, while these measures were developed in concert with the
services, they were developed according to the data that were readily
available to the services, rather than using established guidance on
developing performance measures to thoroughly assess FVAP's
effectiveness and efficiency. Due to the lack of data verification,
the inconsistencies we identified in the data submitted to the DOD
Office of Inspector General, and credibility concerns, the extent to
which the data accurately measure the effectiveness or efficiency of
FVAP's efforts is unknown.
When we discussed FVAP's evaluative information with FVAP's new
Director, he noted that he intends to continue improving the methods
that FVAP uses to gather evaluative information. For example, he is
reaching out to obtain evaluative assistance both within DOD and from
outside sources. He also indicated that he intends to add performance
measures to examine the impact of FVAP's efforts and supplement the
evaluative information (e.g., numbers of materials distributed and
VAOs trained) that the agency currently gathers on its processes.
We also found that FVAP is not systematically collecting evaluative
information on all aspects of its VAO training program. While VAO
outreach to potential voters is integral to several of FVAP's efforts
to achieve its mission, FVAP has not formally evaluated its in-person
or online training course that each VAO receives. FVAP does, however,
obtain ad-hoc feedback from VAOs while conducting its in-person
training workshops and through a general feedback question on the
postelection survey to VAOs. In a 2004 report, we noted that training
programs should have an evaluative component to help ensure the
training is effective and improves performance and results. We also
stated that agencies need evaluation processes that systematically
track the cost and delivery of training.[Footnote 11] The cost
associated with completing the online VAO training is primarily
indirect (e.g., personnel time to complete the course). In contrast,
in-person VAO training results in travel costs for the FVAP trainers
and possibly some VAOs who must travel to another location to be
trained. The locations of the 2-hour in-person VAO training further
illustrate the higher cost for the in-person training. In 2008, FVAP
offered in-person VAO training workshops in a variety of locations: 20
in the area around Washington, D.C., 28 in other U.S. locations, and
62 in foreign locations. Without a formal evaluation of the training,
it is not possible to know (1) how well either training method conveys
knowledge to VAOs, (2) steps needed to enhance the training, (3)
whether in-person training provides added benefits to outweigh its
additional costs, and (4) whether new technology--such as Webinars--
could result in more effective, efficient, or timely delivery of the
training. Since the VAO training is given only in even numbered years,
delaying a formal evaluation of the two methods of training until 2012
would mean that there would be no empirical basis for adjusting the
training until 2014.
In discussing potential reasons why some of the concerns about the
evaluative information remain, FVAP's new Director said that these
types of concerns exist in many organizations where the past emphasis
focuses only on whether something was implemented, rather than on
impact. However, in later discussions, FVAP's new Director mentioned
that the agency will perform more reviews of evaluative information in
the future. Continued use of the evaluative information without adding
steps such as prescribing procedures for the self-reporting of
information and conducting non-response analyses could limit (1)
FVAP's ability to enhance the effectiveness and efficiency of its
overall program and (2) DOD's ability to provide a more complete
picture of FVAP's performance, which could impact the quality of
future reports to Congress submitted in response to section 586 of the
National Defense Authorization Act for Fiscal Year 2010.[Footnote 12]
FVAP's Budget Priorities Appear to Be Generally Aligned with Its
Strategic Plan Goals:
While we could not perform detailed analyses because FVAP does not
budget by strategic goals, FVAP's budgets for fiscal years 2005
through 2009 and the goals in its strategic plans for the same period
appear to be generally aligned. FVAP uses major budget categories and
budget directions from the Defense Human Resource Agency. FVAP's
budget averaged approximately $4 million for the 5 years. Table 2
shows the amounts for each fiscal year and the average across the 5
years for each of six major budget categories.
Table 2: FVAP's Budgets for Fiscal Years 2005 through 2009 Broken into
Six Major Categories:
Budget category: Personnel;
Average[A]: $1,822,000;
Fiscal year: 2005: $1,672,000;
Fiscal year: 2006: $1,710,000;
Fiscal year: 2007: $1,800,000;
Fiscal year: 2008: $1,727,000;
Fiscal year: 2009: $2,200,000.
Budget category: Contracts;
Average[A]: $1,665,000;
Fiscal year: 2005: $1,787,000;
Fiscal year: 2006: $1,497,000;
Fiscal year: 2007: $1,642,000;
Fiscal year: 2008: $2,306,000;
Fiscal year: 2009: $1,091,000.
Budget category: Operations;
Average[A]: $234,000;
Fiscal year: 2005: $285,000;
Fiscal year: 2006: $231,000;
Fiscal year: 2007: $176,000;
Fiscal year: 2008: $158,000;
Fiscal year: 2009: $322,000.
Budget category: Travel;
Average[A]: $145,000;
Fiscal year: 2005: $35,000;
Fiscal year: 2006: $220,000;
Fiscal year: 2007: $60,000;
Fiscal year: 2008: $350,000;
Fiscal year: 2009: $60,000.
Budget category: Information technology;
Average[A]: $103,000;
Fiscal year: 2005: $55,000;
Fiscal year: 2006: $125,000;
Fiscal year: 2007: $78,000;
Fiscal year: 2008: $105,000;
Fiscal year: 2009: $150,000.
Budget category: Training;
Average[A]: $33,000;
Fiscal year: 2005: $25,000;
Fiscal year: 2006: $14,000;
Fiscal year: 2007: $15,000;
Fiscal year: 2008: $50,000;
Fiscal year: 2009: $60,000.
Budget category: Total;
Average[A]: $4,001,000;
Fiscal year: 2005: $3,859,000;
Fiscal year: 2006: $3,797,000;
Fiscal year: 2007: $3,771,000;
Fiscal year: 2008: $4,696,000;
Fiscal year: 2009: $3,883,000.
Source: GAO analysis of FVAP data.
Note: Totals may not include congressional adjustments.
[A] The averages in this column are rounded to the nearest thousand
dollars.
[End of table]
According to FVAP's budget requests to Congress for fiscal year 2009,
the program's overall budget generally supports the achievement of its
mission and strategic goals. The strategic plan for 2008 and 2009
contained the following four general goals: (1) encourage adoption of
FVAP legislative initiatives in order to facilitate UOCAVA voting, (2)
maximize use of existing and emerging technologies, (3) improve
marketing and outreach efforts with customers and stakeholders, and
(4) enhance resourcing of FVAP programs.
FVAP's travel budget varies in election years versus non-election
years, and the increased budgets in election years are related to
travel associated with in-person training. In the non-election years
of 2005, 2007, and 2009 when FVAP staff members conducted very few VAO
workshops, the amounts budgeted for travel in those fiscal years were
$35,000, $60,000, and $60,000, respectively. FVAP officials told us
that the travel budget during an odd-numbered fiscal year consists
primarily of staff members' trips to state conferences to meet with
state and local election officials. Such efforts generally correspond
to FVAP's goal 1 objective of engaging the states and territories in
an effort to pass recommended legislative initiatives. In contrast,
the travel budgets for fiscal years 2006 and 2008 were much higher--
$220,000 and $350,000, respectively. The higher amounts reflect FVAP's
traveling for the previously mentioned meetings with election
officials and to locations in the U.S. and foreign countries in order
to conduct VAO training workshops during the fiscal years that
directly precede federal elections. This latter purpose corresponds to
the goal 2 objective to enhance training products and services.
Our analysis of other documents found that FVAP's budget for contracts
also appears to contribute to the agency's efforts to achieve its
strategic goals and objectives. Funding in this category is
particularly important in supporting goal 2 and goal 3 objectives. We
will use the fiscal year 2008 contracts budget to illustrate some
linkages between FVAP's budget and its strategic goals. For example,
* Some of FVAP's contract funds were linked to goal 2 objectives such
as improving the Web site and exploring new technology methods for
Internet voting. For example, in 2008 FVAP used funds from their
contracts budget for the Electronic Transmission Service,[Footnote 13]
absentee ballot tracking equipment for the Military Postal Service
Agency, an electronic voter registration and ballot delivery system,
and redesign of the FVAP Web site. These services and technologies are
used to reach UOCAVA voters and state and local election officials,
FVAP's target populations. FVAP officials told us that some expenses,
such as funds spent on the Electronic Transmission Service and
publication printing (e.g., the Voting Assistance Guide, motivational
posters, and election dates posters), reoccurred between fiscal years
2005 through 2009.
* Funds spent from the contracts budget related to the postelection
surveys generally link to goal 2 and 3 and involve the use of existing
and emerging technologies and the improvement of the program's
marketing and outreach efforts with stakeholders. According to FVAP
officials, these initiatives aid in the design and execution of FVAP's
postelection surveys. Resources for postelection survey planning are
required in the odd-numbered fiscal year prior to a presidential
election and additional resources for planning and executing the
survey and analyzing the results continue into at least the next 2
fiscal years.
Although it was not possible for us to determine how much of the
personnel budget could be linked directly to a specific strategic
goal, our review found that accomplishment of each of FVAP's strategic
objectives required actions by the agency's staff. Among other things,
the first goal of encouraging adoption of FVAP legislative initiatives
is accomplished by FVAP staff developing the initiatives that will
facilitate UOCAVA-related voting and coordinating with state and local
election officials to accept the initiatives. Moreover, the goal 4
objective that is related to employing improved budgeting, staffing,
and training is accomplished by FVAP staff.
FVAP officials told us that the remaining three budget categories--
operations, information technology, and training for FVAP employees--
support FVAP's daily operations. The operations budget includes the
cost of utilities for the program's office space, supplies, and forms
purchases (e.g., Federal Write-in Absentee Ballot and Federal Post
Card Application). Information technology includes basic Web site
expenses and computer equipment purchases. Our analysis and discussion
with FVAP officials revealed that these budget categories provide
resources for FVAP staff to perform their work which helps them to
meet FVAP's strategic goals.
FVAP and USD (P&R) Fully or Partially Implemented Most of the
Recommendations:
FVAP and USD (P&R) fully or partially implemented 17 of 23 DOD Office
of Inspector General and GAO recommendations made from 2001 through
2009.[Footnote 14] The 23 recommendations involve improvements in the
(1) oversight and direction FVAP and USD (P&R) provide to the
services' respective voting assistance programs through formal policy
and implementation guidance and (2) processes for carrying out FVAP's
mission. FVAP and USD (P&R) fully implemented 8 of 16 DOD Office of
Inspector General and GAO recommendations for improving the oversight
and direction that FVAP provides to the services and partially
implemented three of the other eight recommendations. Additionally,
FVAP fully implemented three GAO recommendations for improvements in
the processes it uses to carry out its mission and partially
implemented three of the four remaining recommendations. Table 3
summarizes FVAP's and USD (P&R)'s implementation of DOD Office of
Inspector General's and GAO's recommendations.
Table 3: FVAP's and USD (P&R)'s Implementation of DOD Office of
Inspector General and GAO Recommendations: 2001 through 2009:
Source of recommendation and recommendation status: DOD Office of
Inspector General recommendations: Implemented;
Area of improvement: Oversight and direction to the services: 5;
Area of improvement: FVAP processes: 0;
Area of improvement: Total: 5.
Source of recommendation and recommendation status: DOD Office of
Inspector General recommendations: Partially implemented;
Area of improvement: Oversight and direction to the services: 2;
Area of improvement: FVAP processes: 1;
Area of improvement: Total: 3.
Source of recommendation and recommendation status: DOD Office of
Inspector General recommendations: Not implemented;
Area of improvement: Oversight and direction to the services: 5;
Area of improvement: FVAP processes: 1;
Area of improvement: Total: 6.
Source of recommendation and recommendation status: Subtotal: DOD
Inspector General;
Area of improvement: Oversight and direction to the services: 12;
Area of improvement: FVAP processes: 2;
Area of improvement: Total: 14.
Source of recommendation and recommendation status: GAO
recommendations: Implemented;
Area of improvement: Oversight and direction to the services: 3;
Area of improvement: FVAP processes: 3;
Area of improvement: Total: 6.
Source of recommendation and recommendation status: GAO
recommendations: Partially implemented;
Area of improvement: Oversight and direction to the services: 1;
Area of improvement: FVAP processes: 2;
Area of improvement: Total: 3.
Source of recommendation and recommendation status: GAO
recommendations: Not implemented;
Area of improvement: Oversight and direction to the services: 0;
Area of improvement: FVAP processes: 0;
Area of improvement: Total: 0.
Source of recommendation and recommendation status: Subtotal: GAO;
Area of improvement: Oversight and direction to the services: 4;
Area of improvement: FVAP processes: 5;
Area of improvement: Total: 9.
Source of recommendation and recommendation status: Combined DOD
Inspector General and GAO recommendations: Implemented;
Area of improvement: Oversight and direction to the services: 8;
Area of improvement: FVAP processes: 3;
Area of improvement: Total: 11.
Source of recommendation and recommendation status: Combined DOD
Inspector General and GAO recommendations: Partially implemented;
Area of improvement: Oversight and direction to the services: 3;
Area of improvement: FVAP processes: 3;
Area of improvement: Total: 6.
Source of recommendation and recommendation status: Combined DOD
Inspector General and GAO recommendations: Not implemented;
Area of improvement: Oversight and direction to the services: 5;
Area of improvement: FVAP processes: 1;
Area of improvement: Total: 6.
Source of recommendation and recommendation status: Total: Combined
DOD Inspector General and GAO;
Area of improvement: Oversight and direction to the services: 16;
Area of improvement: FVAP processes: 7;
Area of improvement: Total: 23.
Source: GAO analysis of DOD Office of Inspector General, FVAP, and GAO
data.
[End of table]
FVAP and USD (P&R) Fully or Partially Implemented Most of the
Recommendations to Improve Oversight and Direction:
FVAP and USD (P&R) fully implemented 8 of 16 DOD Office of Inspector
General and GAO recommendations and partially implemented 3
recommendations to improve the oversight and direction FVAP provides
to the services to guide their implementation of service-and
installation-level voting assistance programs. However, 5 of the 16
recommendations have not been implemented. In recent years, several
revisions have been made to DOD Directive 1000.04, Federal Voting
Assistance Program (FVAP), and many of the changes are related to DOD
Office of Inspector General and GAO recommendations. For example, the
directive was revised in 2002 to direct the services to develop
written policies to support all eligible military personnel and their
family members, following a 2001 GAO recommendation that DOD direct
service secretaries to develop voting assistance directives that fully
reflect the requirements of DOD Directive 1000.04. DOD Directive
1000.04 was also revised in 2002 to require the heads of DOD
components to assign one Unit VAO within each unit of 25 or more
permanently assigned members, and was revised again in 2004 to require
assignment of an additional Unit VAO for each additional 50 members
above the 25 member base. Prior to these revisions, we reported that
VAOs were sometimes responsible for too many voters as a result of the
lack of DOD-wide criteria for determining how many voters a VAO should
serve and recommended that DOD consult with the services and revise
DOD Directive 1000.04 to establish recommended ratios of VAOs to
population served. In addition to changes made through the directive,
FVAP and USD (P&R) directed the revision of service-level voting
assistance regulations and provided additional clarifying guidance to
the military services through the biennial FVAP voting action plans
and memorandums. Table 4 summarizes the DOD Office of Inspector
General's and GAO's recommendations to improve oversight of and
direction to the services and lists the actions that FVAP and USD
(P&R) took to implement the recommendations.
Table 4: DOD Office of Inspector General and GAO Recommendations to
Improve Oversight of and Direction to the Services and FVAP's and USD
(P&R)'s Actions Taken to Implement the Recommendations: 2001 through
2009:
Recommendation (source, year): Direct service secretaries to develop
voting assistance directives that fully reflect the requirements of
DOD Directive 1000.04. (GAO, 2001);
Actions taken: Implemented: USD (P&R) revised Directive 1000.04 in
2002 to direct the services to develop written policies to support all
eligible military personnel and their family members.
Recommendation (source, year): Consult with the services and revise
DOD Directive 1000.04 to establish recommended ratios of VAOs to
population served. (GAO, 2001);
Actions taken: Implemented: USD (P&R) revised Directive 1000.04 in
2002 to require that the heads of DOD components assign one unit VAO
within each unit of 25 or more permanently assigned members, and added
language in 2004 stating that an additional unit VAO should be
assigned for each additional 50 members above the 25 member base.
Recommendation (source, year): Actively collect and share best
practices identified by service and Department of State voting
assistance programs. (GAO, 2001);
Actions taken: Implemented: DOD's biennial voting action plans require
FVAP to collect and share best practices identified by the services
and Department of State. FVAP published best practices on its Web site
and in the Voting Information News newsletter that it distributes
monthly to VAOs, and discussed them at voting workshops.
Recommendation (source, year): Direct service secretaries to require
that the senior service voting representatives monitor installations'
voting assistance programs, periodically provide briefings to FVAP on
the services' efforts to comply with DOD policy, and submit a final
report by June 30 of odd-numbered years. (GAO, 2001);
Actions taken: Partially implemented: DOD Directive 1000.04 requires
that the services file an after-action report to FVAP, and the
biennial voting action plans require that these reports be filed by
January 15 of odd-numbered years. In late 2003, the USD (P&R) began
holding monthly meetings with FVAP and the senior service voting
representatives to discuss the status of service voting assistance
programs. Neither the DOD Directive nor the voting action plans
explicitly require the senior service voting representatives to
monitor installation-level voting assistance programs.
Recommendation (source, year): Provide oversight to ensure that
service voting assistance program regulations are consistent with DOD
Directive 1000.04 requirements. (DOD Office of Inspector General,
2003);
Actions taken: Implemented: According to follow up completed by the
Office of Inspector General, USD (P&R) fulfilled this recommendation
by issuing a memorandum to require that the service Secretaries revise
their policies to correct discrepancies DOD Office of Inspector
General found between the policies and DOD Directive 1000.04.
Recommendation (source, year): Expedite the revision and issuance of
DOD Directive 1000.04 and the DOD Voting Action Plan for 2004-2005.
(DOD Office of Inspector General, 2004);
Actions taken: Implemented: DOD issued the revised Directive 1000.04
on April 14, 2004. USD (P&R) and FVAP released the Voting Action Plan
for 2004-2005 on February 11, 2004.
Recommendation (source, year): Provide oversight to ensure that
service voting assistance program regulations and service voting plans
are consistent with the requirements established by the new guidance.
(DOD Office of Inspector General, 2004);
Actions taken: Implemented: FVAP reviewed the revised service policies
to ensure compliance with DOD Directive 1000.04.
Recommendation (source, year): Revise DOD Directive 1000.04 to require
FVAP and the services to collect and analyze metrics on a more
frequent basis. (DOD Office of Inspector General, 2005);
Actions taken: Not implemented: The current version of the draft DOD
Instruction 1000.04, if finalized, will require the director of FVAP
to establish minimum voting program metrics to be used by the DOD
components and the Uniformed Services in evaluating their individual
voting assistance programs, and report on the results of those
metrics.[A]
Recommendation (source, year): Revise DOD Directive 1000.04 to require
that VAO training include all program objectives outlined in DOD
Directive 1000.04 and good ideas and best practices as tools for
accomplishing objectives, and that VAOs should complete FVAP training
within 60 days of their appointment. (DOD Office of Inspector General,
2005);
Actions taken: Not implemented: FVAP collects and shares best
practices, and discusses them at VAO training workshops. However,
based on our observation, VAO training does not address all program
objectives of DOD Directive 1000.04, and the Directive does not
require that all objectives be addressed in the training. The current
version of draft DOD Instruction 1000.04, if finalized, would require
VAOs to complete training within 90 days of their appointment.[A]
Recommendation (source, year): Revise DOD Directive 1000.04 to permit
heads of the DOD components and the services to exercise flexibility
in how many unit VAOs they assign and at what level. (DOD Office of
Inspector General, 2006);
Actions taken: Not implemented: USD (P&R) issued a memorandum to the
Secretaries of the Military Departments stating that the guidance and
direction found in DOD Directive 1000.04, which includes guidance
regarding the ratio of VAOs to assigned members within a unit and the
desired rank or pay grade of VAOs, should be used as a baseline and
may be adjusted to meet specific requirements and cultures within each
service. In addition, the memorandum describes revisions to the
relevant provisions in the current version of the draft DOD
Instruction 1000.04 and emphasizes the flexibility of those
provisions.[A]
Recommendation (source, year): Direct the services to establish a "one-
click" Web site access from its home page to a voting page that
provides service and FVAP-related information and materials. (DOD
Office of Inspector General, 2006);
Actions taken: Partially implemented: The current version of draft DOD
Instruction 1000.04, if finalized, will require each service to have a
direct link from its Web site home page to the service's voting
assistance program home page.
Recommendation (source, year): Publish guidance to the services
clarifying DOD Directive 1000.04 on the commander's flexibility in
establishing the ratio of unit VAOs to unit members. (DOD Office of
Inspector General, 2007);
Actions taken: Implemented: USD (P&R) issued a memorandum to the
Secretaries of the Military Departments stating that the guidance and
direction found in DOD Directive 1000.04, which includes guidance
regarding the ratio of VAOs to assigned members within a unit, should
be used as a baseline and may be adjusted to meet specific
requirements and cultures within each service. In addition, the
memorandum describes revisions to the relevant provisions in the
current version of the draft DOD Instruction 1000.04 and emphasizes
the flexibility of those provisions.[A]
Recommendation (source, year): Revise DOD Directive 1000.04 to provide
flexibility for pay grades required of personnel assigned VAO duties.
(DOD IG, 2007);
Actions taken: Not implemented: USD (P&R) issued a memorandum to the
Secretaries of the Military Departments stating that the guidance and
direction found in DOD Directive 1000.04, which includes guidance
regarding the desired rank or pay grade of VAOs, should be used as a
baseline and may be adjusted to meet specific requirements and
cultures within each service. In addition, the memorandum describes
revisions to the relevant provisions in the current version of the
draft DOD Instruction 1000.04, and emphasizes the flexibility of those
provisions.[A]
Recommendation (source, year): Publish guidance to the services
emphasizing the importance and benefits of appointing civilian
personnel as installation VAOs. (DOD Office of Inspector General,
2007);
Actions taken: Implemented: USD (P&R) issued a memorandum to the
Secretaries of the Military Departments stating that the guidance and
direction found in DOD Directive 1000.04, which says that where
possible, Installation VAOs should be a civilian GS-12 or higher,
should be used as a baseline and may be adjusted to meet specific
requirements and cultures within each service. In addition, the
memorandum describes revisions to the relevant provisions in the
current version of the draft DOD Instruction 1000.04.[A]
Recommendation (source, year): Direct the services to comply with a
recommendation from the 2005 report to establish one-click links from
their home page to its voting assistance site. (DOD Office of
Inspector General, 2007);
Actions taken: Partially implemented: The current version of draft DOD
Instruction 1000.04, if finalized, will require each service to have a
direct link from its Web site home page to the service's voting
assistance program home page.
Recommendation (source, year): Revise DOD Directive 1000.04 to
authorize appointment of civilian personnel as unit VAOs where
feasible. (DOD Office of Inspector General, 2007);
Actions taken: Not implemented: USD (P&R) determined that this
recommendation may not be practicable because civilian government
personnel within the United States may not be authorized by law to
administer oaths.
Source: GAO analysis of DOD Office of Inspector General, USD (P&R),
FVAP, and GAO information.
[A] Although the DOD Instruction 1000.04, which will replace the
current DOD Directive 1000.04, was still in draft during our review,
FVAP officials told us that the services had been informed of the new
provisions in the policy for the purpose of implementing their
respective voting assistance programs.
[End of table]
FVAP and USD (P&R) partially implemented three other recommendations.
In 2001, GAO recommended that the service Secretaries should (1)
require the senior service voting representatives to monitor
installations' voting assistance programs, (2) periodically update
FVAP on efforts to meet the requirements of DOD Directive 1000.04, and
(3) submit a final report by June 30 of odd-numbered years. In
response, USD (P&R) began holding monthly meetings in late 2003 with
FVAP staff and senior service voting representatives to discuss the
status of service voting assistance programs. However, neither DOD
Directive 1000.04 nor FVAP's biennial voting action plans explicitly
required senior service voting representatives to monitor
installations' voting assistance programs. In our 2001 report, we
noted that such monitoring would enable the services to identify and
address weaknesses in their programs. In addition, FVAP partially
addressed a 2006 and 2007 DOD Office of Inspector General
recommendation in its the current draft that includes a requirement
that each service have a direct link from its Web site home page, to
the service's voting assistance program home page.
FVAP did not implement five of the 16 recommendations. While FVAP has
created a draft version of DOD Instruction 1000.04, it is still under
development and has not been finalized. The draft of DOD Instruction
1000.04--if finalized as currently written--would replace the existing
directive and include several new provisions responding to DOD Office
of Inspector General recommendations that call for revisions to
Directive 1000.04. For example, the DOD Office of Inspector General
recommended that DOD revise Directive 1000.04 to require FVAP and the
services to collect and analyze metrics on a more frequent basis.
While FVAP and the services recently developed metrics, and the draft
DOD Instruction, if finalized, would require the director of FVAP to
(1) establish minimum voting program metrics to be used by the DOD
components and the Uniformed Services in evaluating their individual
voting assistance programs, and (2) report on the results of those
metrics, the draft guidance has not been finalized and the directive
has not been revised as recommended. In addition, recommendations were
made to revise DOD Directive 1000.04 to (1) require that VAO training
include all program objectives, (2) permit heads of the DOD components
and the services to exercise flexibility in how many unit VAOs they
assign, and (3) provide more flexibility for pay grades required of
personnel assigned VAO duties. FVAP also did not implement a 2005 DOD
Office of Inspector General recommendation to revise DOD Directive
1000.04 to require that VAO training cover all program objectives in
the directive and best practices, and that the training be completed
within 60 days of a VAO's appointment. The current draft DOD
Instruction 1000.04 if finalized will require that VAO training be
completed within 90 days of an individual's appointment. Further,
while FVAP does collect best practices among VAOs and shares some of
these in VAO training workshops, the training workshops we observed
did not cover all the program objectives from DOD Directive 1000.04.
For example, the workshops did not cover the requirement that the DOD
components and uniformed services are to establish and publicize a
special telephone service, a "Voting Action Line," to link unit voting
officers with their respective Service or Departmental Voting Action
Officer. In addition, FVAP did not implement a 2007 DOD Office of
Inspector General recommendation to revise Directive 1000.04 to
authorize appointment of civilian personnel as unit VAOs in order to
increase continuity in the unit VAO positions.[Footnote 15] In the
written response to this recommendation, USD (P&R) stated that this
recommendation may not be practicable. By way of explanation, it cited
potential limitations on the authority of civilian government
employees within the United States to administer oaths.
FVAP Fully or Partially Implemented All But One Recommendation to
Improve Its Processes:
FVAP fully implemented three of seven recommendations pertaining to
changes in the processes that it uses to execute its mission, and it
partially implemented another three such recommendations. For example,
in 2001 we recommended that FVAP revise its postelection survey
methodology to gather nationally projectable data on disqualified
overseas and military absentee ballots from local election officials.
In response, FVAP redesigned the postelection survey to obtain the
number of absentee ballots from UOCAVA-eligible voters not counted and
the reasons for disqualifying the ballots. At the time of our
recommendation, we concluded that the collection and analysis of this
information over time could be useful in helping assess whether
efforts to improve voting assistance and work with states have a
positive impact on military and overseas citizens being able to vote.
Moreover, such information should help Congress assess the extent to
which military and overseas ballots are disqualified and evaluate
whether a legislative remedy is needed. FVAP also took steps to
implement two GAO recommendations from 2007 to improve the security
and accuracy of its electronic and Internet initiatives. The steps
included (1) the addition of a cautionary statement to FVAP's
electronic blank ballot request and delivery system to warn UOCAVA
voters to remove personal data from their computers and (2) the
development of a process to review online UOCAVA voting guidance at
the FVAP Web site by creating a checklist with steps that FVAP staff
must take when making changes to the Web site. Table 5 summarizes the
DOD Office of Inspector General's and GAO's recommendations to improve
FVAP processes and lists the actions taken by FVAP to implement the
recommendations.
Table 5: DOD Office of Inspector General and GAO Recommendations to
Improve FVAP Processes and Actions Taken by FVAP to Implement the
Recommendations: 2001 through 2009:
Recommendation (source, year): Develop a methodology to gather
nationally projectable data on disqualified military and overseas
absentee ballots and reasons for disqualification. (GAO, 2001);
Action taken: Implemented: FVAP redesigned the methodology of its
postelection survey of local election officials to obtain nationally
projectable data on the number of absentee ballots not counted and the
reasons for the disqualification of absentee ballots from military and
overseas citizens.
Recommendation (source, year): Develop an automated electronic
delivery and reporting system that would be used to disseminate
information to voters. (DOD Office of Inspector General, 2005);
Action taken: Not implemented: According to FVAP officials, they
determined, along with the services, that it was infeasible because
some servicemembers do not have e-mail or Internet access.
Recommendation (source, year): Incorporate lessons learned into plans
for future systems such as those we identified, including adding
cautionary statements to future ballot request and receipt systems to
warn UOCAVA voters to remove personal data from their computers. (GAO,
2007);
Action taken: Implemented: FVAP added a cautionary statement to its
electronic blank ballot request and delivery system.
Recommendation (source, year): Institutionalize a process to review
online UOCAVA guidance to ensure that DOD provides accurate and
consistent information to UOCAVA voters. (GAO, 2007);
Action taken: Implemented: FVAP has developed a process to review
online voting guidance at its Web site by creating a checklist with
steps that FVAP staff must take when changes to the Web site are made.
Recommendation (source, year): Comply with the information security
requirements in the DOD Certification and Accreditation Process
guidance. (GAO, 2007);
Action taken: Partially implemented: As of February 2010, FVAP
officials told us they plan to ensure that the Electronic Transmission
Service for the November 2010 federal election complies with the DOD
requirements. A contract for the service has not yet been awarded.
Recommendation (source, year): Create an integrated, comprehensive,
long-term, results-oriented plan for future electronic voting
programs. (GAO, 2007);
Action taken: Partially implemented: To date, FVAP has developed some
high-level tasks and estimated timeframes for such a program, but FVAP
officials told us they are waiting for the Election Assistance
Commission to issue electronic absentee voting guidelines in order to
finish their plan.
Recommendation (source, year): Develop metrics and standards for
assessing effectiveness and compliance of services' and DOD's Voting
Assistance Programs. (DOD Office of Inspector General, 2008);
Action taken: Partially implemented: FVAP established some performance
metrics for implementation beginning in the 2008 election year, but we
found that some of these metrics have weaknesses.
Source: GAO analysis of DOD Office of Inspector General, FVAP, and GAO
information.
[End of table]
FVAP partially implemented three of the remaining four DOD Office of
Inspector General's and GAO's recommendations for improvements to its
processes. First, in response to a 2008 DOD Office of Inspector
General recommendation that USD (P&R) and the service Secretaries
should develop metrics for assessing their voting assistance programs,
FVAP established some performance metrics for implementation beginning
in the 2008 election year. However, as we noted earlier in our report,
we found that some of FVAP's evaluative information can continue to be
strengthened. Second, FVAP officials told us they cannot develop a
long-term plan for future electronic absentee voting programs, which
we recommended in 2007, without the establishment of electronic
absentee voting guidelines by the Election Assistance Commission.
[Footnote 16] According to a Commission official, voting guidelines
are not yet complete because, among other reasons, National Institute
of Technology officials--who are providing assistance to the
Commission in developing the guidelines--do not believe that a secure
electronic absentee voting system is technologically feasible at this
time. In the interim, however, FVAP has established some preliminary,
high-level tasks and estimated time frames for the development of an
electronic absentee voting system. In our 2007 report, we concluded
that until FVAP develops the type of plan we recommended, DOD is not
in a position to address congressional expectations to establish
secure and private electronic and Internet-based voting initiatives.
Third, FVAP officials told us they will ensure their Electronic
Transmission Service complies with DOD's information security
requirements--as we recommended in 2007--prior to the November 2010
federal election. As we reported in 2008, until FVAP fulfills this
recommendation by performing and documenting security assessments and
certifications for its Electronic Transmission Service, it has not
taken all the necessary measures to secure its system and comply with
DOD's information security requirements.
FVAP did not implement one DOD Office of Inspector General
recommendation for improving its processes. In 2005, DOD Office of
Inspector General recommended that FVAP develop an automated
electronic delivery and reporting system that could be used to
disseminate information to voters via targeted mass electronic
communication and track the extent to which the information had
reached the entire population of servicemembers. FVAP officials told
us they determined that this recommendation is infeasible because not
all servicemembers have ready access to e-mail or the Internet.
Alternatively, FVAP officials told us they have improved the
information available to UOCAVA voters at the FVAP Web site,
disseminated voting assistance alerts through the heads of the
services, and used the comments section on servicemembers' leave and
earnings statements to inform them about absentee voting procedures.
Conclusions:
Members of Congress, advocacy groups, and others have called for
improved absentee-voting opportunities for the estimated 6-million
U.S. citizens covered by UOCAVA while living in other countries. The
Secretary of Defense is the presidential designee with the primary
responsibility for federal UOCAVA functions, and through the Under
Secretary of Defense for Personnel and Readiness (USD (P&R)), the
Federal Voting Assistance Program provides support to potential
overseas absentee voters in order to facilitate their participation in
federal elections. Toward that end, FVAP took many actions to (1)
address the requirements identified in DOD Directive 1000.04, (2)
implement recommendations that DOD Office of Inspector General and GAO
made to improve FVAP's oversight of and direction to the services as
well as to enhance its own processes, and (3) evaluate its efforts to
accomplish its mission. Program evaluation is an area where FVAP can
continue to improve. Even though FVAP added new performance measures
and more rigor for some evaluative methods during recent years, our
analyses identified some continuing challenges that FVAP faces in
obtaining reliable, consistent, and credible evaluative information
for monitoring and adjusting its efforts to address requirements in
the DOD directive. Our work showed that these challenges are the
result of such things as not following survey-related guidance from
the Office of Management and Budget, gathering data on credible but
limited content issues, and not checking the reliability of self-
reported information. The new FVAP Director has reached out to sources
in DOD and externally to obtain evaluative expertise that FVAP has not
previously used, but the outreach has been too recent to determine its
impact on FVAP's evaluative processes. However, delays in improving
the evaluation of FVAP activities could result in missed opportunities
to both improve the data that FVAP will have following the 2010
federal election and make data-guided improvements before the 2012
election. Delay in improving the evaluation of the VAOs' training is
particularly important because VAOs occupy such a critical role in
helping FVAP to address the directive-specified requirements. For
example, the absence of data comparing the performance of VAOs who had
online versus in-person training raises questions about:
* the relative effectiveness of the two training methods,
* whether the online training produces enough added benefit to warrant
its much higher cost than remote training, and:
* whether newer technological alternatives should replace one or both
of the current training methods.
* If FVAP were to address these areas that need improvements promptly,
the agency would also have better data to provide to Congress in
response to the new requirement for an annual report which was
mandated in the National Defense Authorization Act for Fiscal Year
2010.
Recommendations for Executive Action:
To improve the effectiveness of DOD's absentee voting assistance
efforts, we recommend that the Secretary of Defense direct the Under
Secretary of Defense for Personnel and Readiness to take the following
three actions:
* Assess the methodologies currently in use to gather evaluative
information on FVAP's efforts to address the requirements in DOD
Directive 1000.04.
* Implement, where needed, improved and supplemental program
evaluation methodologies to address concerns such as those that we
identified in this report.
* Evaluate current and alternative methods for training VAOs as part
of federal elections.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, DOD concurred with two
of our recommendations and partially concurred with a third
recommendation. The department's written comments are reprinted in
appendix III.
DOD concurred with our recommendation to assess the methodologies
currently in use to gather evaluative information on FVAP's efforts to
address the requirements in DOD Directive 1000.04. DOD stated that it
is already developing, with the military services, new metrics to
evaluate the program's effectiveness in meeting the requirements of
DOD Directive 1000.04, as well as the new responsibilities identified
in the Military and Overseas Voters Empowerment Act. While developing
these metrics is a step in the right direction, it is too soon to
determine how well and fully the new metrics will evaluate the
program's effectiveness to meet its requirements.
DOD partially concurred with our recommendation to implement, where
needed, improved and supplemental program-evaluation methodologies to
address concerns such as those that we identified in this report. DOD
discussed its survey methodologies only and therefore did not indicate
how FVAP might (1) address the non-survey concerns that we identified
with its measures of success and (2) supplement its current
methodologies to enhance the agency's program evaluation efforts. For
instance, FVAP did not identify plans to conduct reliability
assessments or other evaluations to promote data consistency and
accuracy. With regard to its surveys, DOD stated that the
"exceptionally low" response rate for the overseas civilian survey
highlights the problems in defining the actual size and composition of
the overseas American population. Given the very low response rate,
DOD said FVAP is not considering any of the results of that survey in
its post-2008 elections report to Congress. Additionally, DOD said
that FVAP and the Defense Manpower Data Center are, however,
attempting to refine and improve that survey and will include
nonresponse analysis for its 2010 survey. DOD further agreed with
GAO's concern regarding survey content and reported that FVAP has been
in close discussions with the Defense Manpower Data Center to include
appropriate questions in the 2010 Post-Election Survey. DOD also noted
that it does not believe the post-election surveys of active duty
military members and the local election officials suffer in the same
way as other surveys do from low response rates. DOD added that a
lower response rate is not indicative of a flawed study, nor does the
lack of a specific nonresponse bias analysis indicate that the
original survey results are not statistically valid. The department
supported its assertions by noting that the Defense Manpower Data
Center has much data on the demographics of the people that it surveys
and by stating that an academic researcher found "no strong empirical
relationship between response rates and nonresponse bias." Additional
information needs to be considered in evaluating DOD's assertions.
First, GAO raised the issue of nonresponse bias analysis because 2006
OMB-issued guidance for executive branch agencies states "Agencies
must design the survey to achieve the highest practical rates of
response, commensurate with the importance of survey uses, respondent
burden, and data collection costs, to ensure that survey results are
representative of the target population so that they can be used with
confidence to inform decisions. Nonresponse bias analyses must be
conducted when unit or item response rates or other factors suggest
the potential for bias to occur."[Footnote 17] Thus, DOD's position
that nonresponse bias analysis is not warranted is contrary to an OMB-
prescribed standard for data collection. Second, if the Defense
Manpower Data Center does not use its extensive demographic data to
analyze differences in response rates between groups, DOD cannot draw
conclusions about how well the demographics of the respondents and
nonrespondents actually agreed. Third, DOD's summarization of the
academic researcher's findings is somewhat misleading.[Footnote 18]
The researcher said that the relationship between nonresponse rate and
bias was "modest." More importantly, other points in the researcher's
study noted that: "A synthesis of research studies estimating
nonresponse bias shows the bias often present," and that: "As response
rates decline, researchers face a growing obligation to mount
nonresponse bias studies in order to inform the evaluation of survey
estimates." We reiterate our call for FVAP and Defense Manpower Data
Center to follow OMB guidance to conduct the nonresponse bias
analysis. Continued use of the evaluative information without quality-
assurance steps such as assessing data reliability and potential
nonresponse bias could limit DOD's ability to provide a more complete
picture of FVAP's performance, which could impact the quality of
future reports to Congress submitted in response to section 586 of the
National Defense Authorization Act for Fiscal Year 2010.
DOD concurred with our recommendation to evaluate current and
alternative methods for training VAOs as part of federal elections.
DOD stated that it is pursuing a different approach by transforming
voting assistance into a direct-to-the-voter assistance program of
centralized online tools, training, and assistance provided by FVAP.
For the 2010 election cycle, FVAP said it is most prudent to continue
the traditional VAO program and supplement it with direct-to-the-voter
assistance. With regard to the focus of our recommendation on the
evaluation of VAO training, FVAP indicated that it is implementing
post-training evaluation of its VAO workshops and will include more
detailed questions in its FVAP's surveys of voters and VAOs following
the 2010 election. Reliance on findings from those post-election
surveys is problematic because of concerns such as response rates and
identification of the population. Additional evaluation methods could
be useful, such as testing samples of participants on how much
knowledge they gained from the training. Similarly, structured
gathering of participants' perceptions (e.g., which training topics
need additional explanation) and analysis of that information could
offer insights regarding the relative advantages of current and
alternative methods of VAO training.
As we agreed with your office, unless you publicly announce the
contents of this report earlier, we plan no further distribution of it
until 30 days from its date. At that time, we will send copies to
other interested parties. The report will also be available at no
charge on the GAO Web site at [hyperlink, http://www.gao.gov]. Should
you or your staff have any questions about this report, please contact
me at (202) 512-8246. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. Key contributors to this report are listed in appendix IV.
Signed by:
Jack E. Edwards:
Director, Defense Capabilities and Management:
List of Requesters:
The Honorable Robert Brady:
Chairman:
The Honorable Daniel Lungren:
Ranking Member:
Committee on House Administration:
House of Representatives:
The Honorable Susan Davis:
Chairwoman:
Subcommittee on Military Personnel:
Committee on Armed Services:
House of Representatives:
The Honorable Mike Honda:
House of Representatives:
The Honorable Carolyn Maloney:
House of Representatives:
[End of section]
Appendix I: Scope & Methodology:
To evaluate the extent to which the Federal Voting Assistance Program
(FVAP) has addressed its mission and evaluated the effectiveness of
its efforts to conduct that mission, we reviewed and analyzed relevant
laws (e.g., Uniformed and Overseas Citizens Absentee Voting Act--
UOCAVA[Footnote 19]), directives (e.g., Department of Defense (DOD)
Directive 1000.04), and other guidance (e.g., memorandums). We
reviewed UOCAVA and DOD Directive 1000.04 to identify specific federal
responsibilities for absentee voting and compared these
responsibilities with actions taken by FVAP. We also reviewed relevant
reports prepared by GAO, FVAP, DOD Office of Inspector General,
Election Assistance Commission, Overseas Vote Foundation, and Pew
Charitable Trusts to gain a thorough understanding of prior reviews
conducted on FVAP and the challenges involved in the absentee voting
process. To assess the effectiveness of FVAP's efforts to conduct its
mission, we reviewed FVAP's data gathering procedures and available
measures obtained from FVAP and published reports (e.g., ours and
those of the DOD Office of Inspector General and non-governmental
organizations). We then compared FVAP's procedures and measures
against criteria on performance measurement, program evaluation, and
related issues that GAO and others published in reports.[Footnote 20]
We additionally reviewed the methodology and questions used in FVAP's
2008 postelection surveys of servicemembers, VAOs, overseas citizens,
and local election officials and compared FVAP-related information
against criteria in guidance from the Office of Management and Budget.
[Footnote 21] We interviewed key survey program officials from the
Defense Manpower Data Center to obtain information and methodologies
for FVAP's postelection surveys. We also interviewed officials from
FVAP, the Department of State, the Election Assistance Commission,
Overseas Vote Foundation, and Pew Charitable Trusts to obtain their
perspectives on FVAP's efforts. Additionally, we reviewed FVAP's 2010-
2011 Voting Assistance Guide and the agency's Web site to determine
the types of information provided to UOCOVA-covered citizens
considering or attempting to vote absentee. To specifically assess the
requirement that FVAP train VAOs, we held interviews with key
officials and with a nonprobability sample of VAOs from the services
to obtain their perspective of FVAP's VAO training. Because this was a
nonprobability sample of VAOs, the results of these interviews are not
generalizeable to the population of VAOs. To select the VAOs, we
obtained a list of VAOs for the most recent voting cycle from the
Army, Navy, Marine Corps, and Air Force and generated a non-
probability random sample from each list. We observed in-person VAO
training workshops and completed the online training that can also be
used for VAO training. For both types of training, we compared the
methodological and content-related information against requirements
specified in DOD Directive 1000.04 and criteria specified in training
standards that we published.[Footnote 22]
To evaluate the extent to which FVAP has processes in place to align
its budget priorities with its strategic plans and goals, we reviewed
FVAP's budget development procedures and compared its budgets for
fiscal years 2005 through 2009 with the goals described in its
strategic plan for 2008 through 2009. We began with fiscal year 2005
because this is when FVAP's budget reporting requirement shifted from
the Washington Headquarters Services to the Defense Human Resource
Activity. Additionally, we interviewed key FVAP and Defense Human
Resource Activity officials to clarify the program's planning and
budgeting processes. After gaining an understanding of the budget,
budget process, and contents of FVAP's strategic plan, we worked with
FVAP staff to identify which parts of the budget were primarily used
to address each of the goals in the strategic plan.
To determine the extent to which DOD Office of Inspector General and
GAO recommendations made from 2001 through 2009 were implemented by
FVAP, we reviewed DOD Office of Inspector General and GAO reports on
absentee voting to identify any recommendations related to this
program.[Footnote 23] We also interviewed DOD Office of Inspector
General officials who conducted the evaluations of FVAP's and the
Under Secretary of Defense for Personnel and Readiness's
implementation efforts and reviewed documentation on those efforts.
Additionally, we discussed our findings with DOD Office of Inspector
General officials to clarify the recommendations' status, actions
taken by FVAP and the Under Secretary, and any positive outcomes from
implementation. Similarly, we reviewed our reports with UOCAVA-related
recommendations as well as information later obtained to document the
efforts to implement our recommendations. In cases where a
recommendation was not implemented or partially implemented, we sought
additional information from FVAP officials, for example, confirmation
that our finding was correct and the reason why the recommendation was
not fully implemented. For each of our objectives, we assessed the
reliability of the data we analyzed by reviewing existing
documentation related to the data sources and interviewing
knowledgeable agency officials about the data that we used. We found
the data sufficiently reliable for the purposes of this report.
We conducted this performance audit from June 2009 through May 2010 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
[End of section]
Appendix II: DOD Office of Inspector General and GAO Reports
Containing DOD Voting Assistance Program Recommendations:
Department of Defense, Office of the Inspector General. 2008
Evaluation of the DOD Voting Assistance Program. Report Number IE-2009-
005. Arlington, Virginia: April 30, 2009.
Department of Defense, Office of the Inspector General. 2007
Evaluation of the Federal Voting Assistance Program in the Department
of Defense. Report Number IE-2008-002. Arlington, Virginia: March 31,
2008.
GAO. Elections: Action Plans Needed to Fully Address Challenges in
Electronic Absentee Voting Initiatives for Military and Overseas
Citizens. [hyperlink, http://www.gao.gov/products/GAO-07-774].
Washington, D.C.: June 14, 2007.
Department of Defense, Office of the Inspector General, 2006
Evaluation of the Federal Voting Assistance Program in the Department
of Defense. Report Number IE-2007-004. Arlington, Virginia: March 31,
2007.
Department of Defense, Office of the Inspector General, Evaluation of
the Voting Assistance Program. Report Number IE-2006-001. Arlington,
Virginia: March 31, 2006.
Department of Defense, Office of the Inspector General, Evaluation of
the Voting Assistance Program. Report Number IE-2005-001. Arlington,
Virginia: March 31, 2005.
Department of Defense, Office of the Inspector General, DOD
Implementation of the Voting Assistance Program. Report Number D-2004-
065. Arlington, Virginia: March 31, 2004.
Department of Defense, Office of the Inspector General, DOD Compliance
With the Uniformed and Overseas Citizens Absentee Voting Act. Report
Number D-2003-072. Arlington, Virginia: March 31, 2003.
GAO. Elections: Voting Assistance to Military and Overseas Citizens
Should Be Improved. [hyperlink,
http://www.gao.gov/products/GAO-01-1026]. Washington, D.C.: September
28, 2001.
[End of section]
Appendix III: Comments from the Department of Defense:
Under Secretary Of Defense:
Personnel And Readiness:
4000 Defense Pentagon:
Washington, D.C. 20301-4000:
May 24, 2010:
Mr. Jack E. Edwards:
Director, Defense Capabilities and Management:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Mr. Edwards:
This is the Department of Defense (DoD) response to the GAO draft
report, Elections: DoD Can Strengthen Evaluation of Its Absentee
Voting Assistance Program, dated April 20, 2010 (GAO Code 351362/GA0-
10-476).
The Federal Voting Assistance Program (FVAP) will make strategic and
operational improvements based on the findings of its 2008 post-
election surveys of Active Duty Military and Local Election Officials,
customer and stakeholder feedback, as well as input from the GAO and
DoD Inspector General reports. GAO only had the 2008-09 FVAP Strategic
Plan available for analysis. FVAP has since initiated a strategic
planning effort based upon new ideas for voting assistance
improvements, the results of the 2008 post-election surveys, and
inputs from your staff during this audit. This new effort will
substantially address the issues raised in this report.
The 2008 post-election surveys show military voter registration rates
exceed those of the national electorate, and that military voter
participation rates, when adjusted for the substantial age and gender
differences between military and civilian populations, also exceed the
national voter participation rates. Where military voter success
differs sharply from the general electorate is in absentee ballot
return rates: only 63% of military absentee ballots were returned for
the November 2008 general election, compared to 91% of the general
electorate. It appears the primary reason for this failure is that
ballots were not sent to military voters far enough before the
election for military voters to receive, complete, and return them by
State-imposed absentee ballot receipt deadlines. This is outside the
control of the military Voting Assistance Officers, of course, but
significantly impacts overall military voter success rates. The
requirement imposed upon States by the MOVE Act to transmit ballots to
military and overseas voters at least 45 days before federal
elections, and to transmit them electronically, should substantially
improve the absentee ballot return rates for military and overseas
voters.
We believe the FVAP surveys show the FVAP program is successfully
assisting military voters in exercising their right to vote. We agree
we can do even better, and are committed to further studies to monitor
and improve what we do. Thank you for your cooperation and the
opportunity to comment on this report.
Sincerely,
Signed by:
Clifford L. Stanley:
Attachment: Comments to the Recommendations:
[End of letter]
GAO Draft Report ” Dated April 20, 2010:
GAO Code 351362/GAO-10-476:
"Elections: DoD Can Strengthen Evaluation of Its Absentee Voting
Assistance Program"
Department Of Defense Comments To The Recommendations:
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Under Secretary of Defense for Personnel and Readiness
(USD(P&R)) to assess the methodologies currently in use to gather
evaluative information on Federal Voting Assistance Program's (FVAP's)
efforts to address the requirements in DoD Directive 1000.04.
DOD Response: Concur. The Federal Voting Assistance Program (FVAP) is
already developing, with the military Services, the new metrics to
evaluate the program's effectiveness in meeting the requirements of
DoD Directive 1000.04, to include the new responsibilities put forward
in the Military and Overseas Voters Empowerment Act. As noted in the
GAO report, the current draft of the revised DoD Instruction 1000.04
will address many of the shortcomings GAO identified in the prior year
program evaluation information. Further, the combined Unit Voting
Assistance Officer and direct-to-the-voter assistance program FVAP has
undertaken during the 2010 election cycle will present an opportunity
to evaluate both programs against each other.
Recommendation 2: The GAO recommends that the Secretary of Defense
direct the Under Secretary of Defense for Personnel and Readiness
(USD(P&R)) to implement, where needed, improved and supplemental
program evaluation methodologies to address concerns such as those
that we identified in this report.
DOD Response: Partially Concur. The exceptionally low response rate
for the overseas civilian survey highlights the problems the Federal
Voting Assistance Program (FVAP) and others in both the overseas
community and federal government (Department of State and Census
Bureau) face in defining the actual size and composition of the
overseas American population. Given that very low response rate, FVAP
is not considering any of the results of that survey in its post-2008
election report. This is a departure from previous FVAP practice which
would report such survey results, regardless of the survey population
coverage or response rates, no matter how low.
To determine possible remedies, FVAP and the Defense Manpower Data
Center are examining a number of alternative methods to refine and
improve the overseas civilian survey, including revised survey methods
and new outreach programs through overseas citizen advocacy groups.
Regardless, until the overall overseas, non-federal, U.S. citizen
population is better defined, it will be impossible to fully evaluate
the effectiveness of the voting assistance program provided to
overseas Americans; voter registration and voter participation rates
are impossible to calculate if the denominator of the total overseas
American population is unknown.
In contrast, FVAP does not believe the active duty military and the
local election official post-election surveys suffer in the same way
from lower response rates. The Department understands GAO's concerns
with response rates and lack of non-response bias studies during the
2008 Post-Election Survey, but a lower response rate, in and of
itself, is not indicative of a flawed study, nor does the lack of
specific non-response analysis indicate that the original survey
results are not statistically valid. Groves (2006) finds no strong
empirical relationship between response rates and nonresponse bias.
[Footnote 24]
FVAP believes that the active duty survey likely has lower nonresponse
bias than surveys with much higher response rates because generally
survey organizations know very little about survey nonrespondents. For
instance, in telephone surveys, for 'ring-no-answer' cases the
surveyor may only know limited geographic data based on the telephone
exchange. For household interview surveys, the surveyor may only have
outdated knowledge (usually Census data) of block characteristics
(e.g., percent Hispanic). For the FVAP survey of active duty military,
DMDC has exceptionally high quality and timely demographic data
available for both respondents and nonrespondents, including Service,
paygrade, age, gender, education, deployment history, and much more.
DMDC statisticians use knowledge of nonrespondents' characteristics to
create sophisticated statistical adjustments designed specifically to
reduce nonresponse bias, including non-response and post-
stratification weighting adjustments.
For the FVAP survey of local election officials (LEO), the 55 percent
response rate is generally considered high for a survey that does not
include high-cost survey methods like face-to-face interviewing. For
the LEO survey, we also have considerable information on nonrespondent
LEOs, including state, jurisdiction size in number of registered
voters, and type of governmental unit (county versus local). These
data were used both for statistical imputations for item-missing data
and weighting adjustments to compensate for unit nonresponse, and
again both these procedures reduce the nonresponse bias in survey
estimates.
For both these surveys, FVAP and DMDC's statisticians consider
estimates to be representative of their respective populations and,
therefore, the results can be used in program evaluation, policy
decisions, and program planning and execution.
Regardless, the Department will include non-response analysis in its
2010 survey. The Department also agrees with GAO's specific concern
regarding survey content, such as local election official views on the
effectiveness of FVAP's Legislative Initiatives program, and has been
in close discussions with the Defense Manpower Data Center to include
such questions in the 2010 Post-Election Survey.
Recommendation 3: The GAO recommends that the Secretary of Defense
direct the Under Secretary of Defense for Personnel and Readiness
(USD(P&R)) to evaluate current and alternative methods for training
voting assistance officers (VAOs) as part of federal elections.
DOD Response: Concur. The Federal Voting Assistance Program (FVAP)
also agrees with the Department of Defense Inspector General's 2005
assessment of the Voting Assistance Officer program: "voting
assistance will always be a secondary duty [for military unit Voting
Assistance Officers, and] senior leadership can expect improvement
only if a radically different approach is applied."[Footnote 25] To
that end, FVAP is pursuing a different approach by transforming voting
assistance into a direct to the voter assistance program of
centralized online tools, training, and assistance provided by FVAP.
However, for the 2010 election cycle, given the apparent performance
based success of the Voting Assistance Officer program,[Footnote 26]
FVAP felt it most prudent to continue the traditional VAO program and
layer the additional direct-to-the-voter assistance on top of that
unit-and installation-level VAO assistance, in order to evaluate both
against each other. Additionally, the Military and Overseas Voter
Empowerment Act requires extensive voting assistance at installation
voting assistance offices, including additional voting assistance
through the National Voter Registration Act at every installation.
Coupled with increased help desk assistance, the FVAP should begin to
decrease the demands on the Unit VAOs to only those most in need of
assistance.
Regardless, based on GAO's initial communications, FVAP is already
implementing post-training evaluation of VAO workshops, and will
include more detailed contextual and evaluative questions in its voter
and Voting Assistance Officer 2010 Post-Election Surveys.
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
Contact:
Jack Edwards, (202) 512-8246 or EdwardsJ@gao.gov:
Acknowledgments:
In addition to the individual named above, Carleen Bennett, Assistant
Director; Bonita Anderson; Melissa Blanco; Chanee Gaskin; Nicole
Harms; Mae Jones; Jennifer Madison; Amanda Miller; Terry Richardson;
Michael Silver; Susan Tindall; and Yong Song made key contributions to
this report.
[End of section]
Related GAO Products:
Elections: Action Plans Needed to Fully Address Challenges in
Electronic Absentee Voting Initiatives for Military and Overseas
Citizens. [hyperlink, http://www.gao.gov/products/GAO-07-774].
Washington, D.C.: June 14, 2007.
Elections: All Levels of Government Are Needed to Address Electronic
Voting System Challenges. [hyperlink,
http://www.gao.gov/products/GAO-07-576T]. Washington, D.C.: March 7,
2007.
Elections: DOD Expands Voting Assistance to Military Absentee Voters,
but Challenges Remain. [hyperlink,
http://www.gao.gov/products/GAO-06-1134T]. Washington, D.C.: September
28, 2006.
Elections: The Nation's Evolving Election System as Reflected in the
November 2004 General Election. [hyperlink,
http://www.gao.gov/products/GAO-06-450]. Washington, D.C.: June 6,
2006.
Elections: Absentee Voting Assistance to Military and Overseas
Citizens Increased for the 2004 General Election, but Challenges
Remain. [hyperlink, http://www.gao.gov/products/GAO-06-521].
Washington, D.C.: April 7, 2006.
Election Reform: Nine States' Experiences Implementing Federal
Requirements for Computerized Statewide Voter Registration Lists.
[hyperlink, http://www.gao.gov/products/GAO-06-247]. Washington, D.C.:
February 7, 2006.
Elections: Views of Selected Local Election Officials on Managing
Voter Registration and Ensuring Eligible Citizens Can Vote.
[hyperlink, http://www.gao.gov/products/GAO-05-997]. Washington, D.C.:
September 27, 2005.
Elections: Federal Efforts to Improve Security and Reliability of
Electronic Voting Systems Are Under Way, but Key Activities Need to Be
Completed. [hyperlink, http://www.gao.gov/products/GAO-05-956].
Washington, D.C.: September 21, 2005.
Elections: Additional Data Could Help State and Local Elections
Officials Maintain Accurate Voter Registration Lists. [hyperlink,
http://www.gao.gov/products/GAO-05-478]. Washington, D.C.: June 10,
2005.
Department of Justice's Activities to Address Past Election-Related
Voting Irregularities. [hyperlink,
http://www.gao.gov/products/GAO-04-1041R]. Washington, D.C.: September
14, 2004.
Elections: Electronic Voting Offers Opportunities and Presents
Challenges. [hyperlink, http://www.gao.gov/products/GAO-04-975T].
Washington, D.C.: July 20, 2004.
Elections: Voting Assistance to Military and Overseas Citizens Should
Be Improved. [hyperlink, http://www.gao.gov/products/GAO-01-1026].
Washington, D.C.: September 28, 2001.
Elections: The Scope of Congressional Authority in Election
Administration. [hyperlink, http://www.gao.gov/products/GAO-01-470].
Washington, D.C.: March 13, 2001.
[End of section]
Footnotes:
[1] Pub. L. No. 99-410 (1986), codified as subsequently amended at 42
U.S.C. §§1973ff et seq.
[2] GAO, Tax Administration: IRS Needs to Further Refine Its Tax
Filing Season Performance Measures, [hyperlink,
http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: Nov. 22,
2002).
[3] Office of Management and Budget, Guidance on Agency Survey and
Statistical Information Collection, (Washington, D.C.: January 2006).
[4] The National Defense Authorization Act for Fiscal Year 2002, Pub.
L. No. 107-107, §1604 (2001) directed the Secretary of Defense to
carry out a demonstration project under which absent uniformed
services voters are permitted to cast ballots through an electronic
voting system. Initially, the project was to be carried out in the
general election for federal office in either November 2002 or
November 2004. The Ronald W. Reagan National Defense Authorization Act
for Fiscal Year 2005, Pub. L. No. 108-375, §567 (2004) amended that
requirement by requiring the project to be implemented during the
first general election for federal office that occurs after the
Election Assistance Commission establishes electronic absentee voting
guidelines and certifies that it will assist the Secretary of Defense
in carrying out the project. In 2007, we reported that DOD had not
moved forward with the project because the Commission had not
developed the guidelines that DOD needs to develop the demonstration
project.
[5] 42 U.S.C. §1973ff, et seq.
[6] DOD Directive 1000.04, Federal Voting Assistance Program (FVAP)
(Apr. 14, 2004).
[7] GAO, Managing for Results: Critical Issues for Improving Federal
Agencies' Strategic Plans, [hyperlink,
http://www.gao.gov/products/GAO/GGD-97-180] (Washington, D.C.: Sept.
16, 1997).
[8] GAO, Program Evaluation: An Evaluation Culture and Collaborative
Partnerships Help Build Agency Capacity, [hyperlink,
http://www.gao.gov/products/GAO-03-454] (Washington, D.C.: May 2,
2003).
[9] DOD Office of Inspector General, 2008 Evaluation of the DOD Voting
Assistance Program, Report No. IE-2009-005 (Arlington, Va.: Apr. 30,
2009).
[10] Office of Management and Budget's January 2006 Guidance on Agency
Survey and Statistical Information Collections says, among other
things, that information collection requests for "surveys with
expected response rates lower than 80 percent need—a description of
plans to evaluate nonresponse bias." In explaining the reason for such
analyses, the guidance notes that the lower the response rates are and
the greater the differences between respondents and nonrespondents,
the greater the nonresponse bias.
[11] GAO, Human Capital: A Guide for Assessing Strategic Training and
Developmental Efforts in the Federal Government, [hyperlink,
http://www.gao.gov/products/GAO-04-546G] (Washington, D.C.: March
2004).
[12] National Defense Authorization Act for Fiscal Year 2010, Pub. L.
No. 111-84, §586 (2009), requires the presidential designee to submit
an annual report to the President and to the relevant committees of
Congress. That report is to include an assessment of the effectiveness
of activities carried out under 42 U.S.C. §1973ff-2B, including the
activities and actions of FVAP, a separate assessment of voter
registration and participation by overseas voters who are not members
of the uniformed services, and a description of the cooperation
between states and federal governments in carrying out section 1973ff-
2B. Additionally, the report is to contain a description of the
utilization of voter registration assistance under 10 U.S.C. §1533a,
including a description of the specific programs implemented by each
military department of the Armed Forces pursuant to section 1533a and
the number of absent uniformed services voters who utilized voter
registration assistance provided under that section.
[13] The Electronic Transmission Service is a resource that allows fax-
to-email and email-to-fax conversion of election materials during
transmission between state and local election officials and UOCAVA
voters.
[14] See appendix II for DOD Office of Inspector General and GAO
reports reviewed from 2001 to 2009 related to these recommendations.
[15] In general, civilians serve for a longer period of time than
servicemembers before moving to a different unit.
[16] The National Defense Authorization Act for Fiscal Year 2002, Pub.
L. No. 107-107, §1604 (2001) directed the Secretary of Defense to
carry out a demonstration project under which absent uniformed
services voters are permitted to cast ballots through an electronic
voting system. Initially, the project was to be carried out in either
the general election for federal office in either November 2002 or
November 2004. The Ronald W. Reagan National Defense Authorization Act
for Fiscal Year 2005, Pub. L. No. 108-375, §567 (2004) amended that
requirement by requiring the project to be implemented during the
first general election for federal office that occurs after the
Election Assistance Commission establishes electronic absentee voting
guidelines and certifies that it will assist the Secretary of Defense
in carrying out the project. In 2007, we reported that DOD had not
moved forward with the project because the Commission had not
developed the guidelines that DOD needs to develop the demonstration
project.
[17] Office of Management and Budget, Standards and Guidelines for
Statistical Surveys (September 2006).
[18] Robert M. Groves, "Nonresponse Rates and NonResponse Bias in
Household Surveys," Public Opinion Quarterly, Vol. 70, No. 5, Special
Issue, pp. 646-675 (2006). The cited quotes are on pages 646, 657, and
659.
[19] Pub. L. No. 99-410 (1986), codified at 42 U.S.C. §§1973ff et seq.
[20] GAO, Tax Administration: IRS Needs to Further Refine Its Tax
Filing Season Performance Measures, [hyperlink,
http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: Nov. 22,
2002).
[21] Office of Management and Budget, Guidance on Agency Survey and
Statistical Information Collections (Washington, D.C.: January 2006).
[22] GAO, Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government, [hyperlink,
http://www.gao.gov/products/GAO-04-546G] (Washington, D.C.: Mar. 1,
2004).
[23] The DOD Office of Inspector General reports were completed in
response to 10 U.S.C. §1566.
[24] Groves, Robert M. "Nonresponse Rates and Nonresponse Bias in
Household Surveys." Public Opinion Quarterly, 70(5):646-675.
[25] DoD IG, 2004 Evaluation, p. 17, 26.
[26] For the 2008 November general election, 77% of the military was
registered to vote as compared to 71% of the national Citizen Age
Voting Population. When adjusted for the substantial age and gender
differences between the military and the general electorate, 73% of
the military Citizen Voting Age Population voted as compared to 63.6%
of the national Citizen Voting Age Population.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
