Veterans Affairs
Improved Planning Needed to Guide Development and Implementation of Education Benefits System Gao ID: GAO-07-1045 July 31, 2007The Department of Veterans Affairs (VA) administers programs that provide educational resources to veterans, service members, reservists, and eligible family members of veterans. Under an initiative that the Veterans Benefits Administration (VBA) began in 1999, called The Education Expert System (TEES), VA is moving its education benefits claims processing systems to new technology platforms and replacing the current systems with a new system that is to add more automated processes and eliminate most human intervention. GAO was requested to (1) report on the results of TEES development to date and identify the remaining work to complete the project, (2) identify VBA's plans for completing and implementing TEES, and (3) assess the feasibility of developing and deploying TEES before the completion of VBA's Finance and Accounting System. To accomplish these objectives, GAO analyzed relevant documentation, interviewed program and project officials, and observed processing systems at VA regional offices.
To date, as part of the TEES project, VBA has enhanced education benefits claims processing by developing certain capabilities that allow information to be captured in an electronic format. However, VBA has not taken action on various other initiatives that it identified as necessary to complete the project. These initiatives include (1) moving the functionality provided by its aging, stand-alone system used to process flight school, on-the-job training, and apprenticeship claims to a platform compliant with the department's enterprise architecture; (2) moving the processing and payment functionality used for many of the education claims from its current antiquated system to new technology; and (3) creating a rules-based claims processing system that will process all education benefits and types of training. VBA does not have an integrated project management plan for the TEES initiative. According to agency officials, a project management plan originally developed in 2001 has not been updated since 2004. Because VBA does not have an integrated project management plan, it lacks critical elements needed to effectively guide the project to completion, such as a full description of its scope; identification of stakeholders' responsibilities, documented risks, and performance metrics; and an overall approach for coordinating its various education claims improvement initiatives. Moreover, without an integrated project management plan, VBA risks initiating work that could be duplicative and overlapping. VBA's Finance and Accounting System (FAS) is substantially complete and can currently pay new compensation claims; all functionality for compensation and pension payments is scheduled to be completed by August 2008. Thus, since much of the functionality for FAS has been developed, VBA could implement a rules-based system that integrates with FAS. However, while the current strategy calls for using FAS as the payment system for TEES, the existing functional requirements for an interface between the two systems, which were developed in 2003, have not been assessed and updated. Subject matter experts needed to define business requirements for interfacing the systems have not been assigned by a key stakeholder. As a result, it is uncertain when VBA will validate and update functional requirements for the FAS interface with TEES.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-1045, Veterans Affairs: Improved Planning Needed to Guide Development and Implementation of Education Benefits System
This is the accessible text file for GAO report number GAO-07-1045
entitled 'Veterans Affairs: Improved Planning Needed to Guide
Development and Implementation of Education Benefits System' which was
released on July 31, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Subcommittee on Economic Opportunity, Committee on
Veterans' Affairs, House of Representatives:
United States Government Accountability Office:
GAO:
July 2007:
Veterans Affairs:
Improved Planning Needed to Guide Development and Implementation of
Education Benefits System:
Veterans' Education Benefits System:
GAO-07-1045:
GAO Highlights:
Highlights of GAO-07-1045, a report to the Subcommitee on Economic
Opportunity, Committee on Veterans‘ Affairs, House of Representatives
Why GAO Did This Study:
The Department of Veterans Affairs (VA) administers programs that
provide educational resources to veterans, service members, reservists,
and eligible family members of veterans. Under an initiative that the
Veterans Benefits Administration (VBA) began in 1999, called The
Education Expert System (TEES), VA is moving its education benefits
claims processing systems to new technology platforms and replacing the
current systems with a new system that is to add more automated
processes and eliminate most human intervention. GAO was requested to
(1) report on the results of TEES development to date and identify the
remaining work to complete the project, (2) identify VBA‘s plans for
completing and implementing TEES, and (3) assess the feasibility of
developing and deploying TEES before the completion of VBA‘s Finance
and Accounting System. To accomplish these objectives, GAO analyzed
relevant documentation, interviewed program and project officials, and
observed processing systems at VA regional offices.
What GAO Found:
To date, as part of the TEES project, VBA has enhanced education
benefits claims processing by developing certain capabilities that
allow information to be captured in an electronic format. However, VBA
has not taken action on various other initiatives that it identified as
necessary to complete the project. These initiatives include (1) moving
the functionality provided by its aging, stand-alone system used to
process flight school, on-the-job training, and apprenticeship claims
to a platform compliant with the department‘s enterprise architecture;
(2) moving the processing and payment functionality used for many of
the education claims from its current antiquated system to new
technology; and (3) creating a rules-based claims processing system
that will process all education benefits and types of training.
VBA does not have an integrated project management plan for the TEES
initiative. According to agency officials, a project management plan
originally developed in 2001 has not been updated since 2004. Because
VBA does not have an integrated project management plan, it lacks
critical elements needed to effectively guide the project to
completion, such as a full description of its scope; identification of
stakeholders‘ responsibilities, documented risks, and performance
metrics; and an overall approach for coordinating its various education
claims improvement initiatives. Moreover, without an integrated project
management plan, VBA risks initiating work that could be duplicative
and overlapping.
VBA‘s Finance and Accounting System (FAS) is substantially complete and
can currently pay new compensation claims; all functionality for
compensation and pension payments is scheduled to be completed by
August 2008. Thus, since much of the functionality for FAS has been
developed, VBA could implement a rules-based system that integrates
with FAS. However, while the current strategy calls for using FAS as
the payment system for TEES, the existing functional requirements for
an interface between the two systems, which were developed in 2003,
have not been assessed and updated. Subject matter experts needed to
define business requirements for interfacing the systems have not been
assigned by a key stakeholder. As a result, it is uncertain when VBA
will validate and update functional requirements for the FAS interface
with TEES.
What GAO Recommends:
To ensure the successful implementation of TEES, GAO is recommending
that the Secretary of Veterans Affairs, among other actions, require
the development of a comprehensive, integrated project plan to
coordinate and manage the initiative. Commenting on a draft of this
report, VA concurred with GAO‘s recommendations and described actions
planned to address them.
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-1045].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Valerie Melvin at (202)
512-6304 or melvinv@gao.gov.
[End of section]
Contents:
Letter:
Conclusions:
Recommendations:
Agency Comments and Our Evaluation:
Appendix I: Veterans Affairs: Improved Planning Needed to Guide
Implementation of The Education Expert System (TEES):
Appendix II: Comments from the Department of Veterans Affairs:
Appendix III: GAO Contact and Staff Acknowledgments:
Abbreviations:
FAS: Finance and Accounting System:
OMB: Office of Management and Budget:
TEES: The Education Expert System:
VA: Department of Veterans Affairs:
VBA: Veterans Benefits Administration:
United States Government Accountability Office:
Washington, DC 20548:
July 31, 2007:
The Honorable Stephanie Herseth Sandlin:
Chair:
The Honorable John Boozman:
Ranking Member:
Subcommittee on Economic Opportunity:
Committee on Veterans' Affairs:
House of Representatives:
Under an initiative called The Education Expert System (TEES), the
Department of Veterans Affairs (VA) is taking steps intended to improve
its education benefits delivery services by moving its education claims
processing systems to new technology platforms and a new architecture.
The effort, which the Veterans Benefits Administration (VBA) initiated
in December 1999, is expected to enable faster and more accurate
processing of education claims by replacing the existing education
benefits systems within VBA's antiquated Benefits Delivery Network with
a new rules-based system that will add more automated capabilities and
eliminate most human intervention. When it began the initiative, VBA
had planned to complete the new system by September 2005; however, in
2004, the department refocused and rebaselined the system's development
effort. VA currently estimates that the TEES initiative will be
completed by 2011.
At your request, we examined VA's actions on the TEES initiative.
Specifically, our objectives were to:
* report the results of VBA's development of TEES to date, and identify
the remaining work needed to complete the system;
* identify VBA's plans for completing and implementing TEES; and:
* assess the feasibility of developing and deploying TEES application
processing functions before the completion of VBA's Finance and
Accounting System (FAS).
To address these objectives, we obtained and analyzed documentation
describing the planned and ongoing activities for TEES and FAS,
including functional requirements documents, system development and
code conversion contracts for the initiative, regional office
processing metrics, and cost information. We supplemented our analysis
with interviews of education program officials, project managers, and
other officials responsible for developing the new system. In addition,
we observed education claims processing and the systems currently used
to support claims processing at two of VA's four regional offices--
Atlanta, Georgia and Muskogee, Oklahoma.[Footnote 1] We performed our
work at VA's headquarters in Washington, D.C., and at the Atlanta and
Muskogee Regional Offices from October 2006 through May 2007, in
accordance with generally accepted government auditing standards.
On May 25, 2007, we provided your offices with briefing slides
(included as appendix I) that outlined the results of our study.
Subsequently, we met with your staff on June 15, 2007, to further
discuss our findings and recommendations. The purpose of this report is
to provide the published briefing slides to you and to officially
transmit our recommendations to the Secretary of Veterans Affairs.
In summary, our study highlighted the following:
* VBA has, to date, enhanced education benefits claims processing by
developing certain functionalities to allow information to be captured
in an electronic format. For example, it has developed automated
systems that allow (1) education institutions to provide online
enrollment certifications, (2) students to provide online and
telephonic verification of enrollment, and (3) the public to inquire
about approved academic programs, licensing and certification programs,
and national exams. However, VBA has not taken action on various other
initiatives that it identified as necessary to complete the new system
and eliminate most human intervention. These initiatives include (1)
moving the functionality provided by its aging, stand-alone system used
to process flight school, on-the-job training, and apprenticeships
claims to a platform compliant with the department's enterprise
architecture; (2) moving the processing and payment functionality used
for many of the education claims from its current antiquated system to
new technology; and (3) creating a rules-based claims processing system
that will process all education benefits and types of training.
* VBA does not have an integrated project plan for the TEES initiative.
According to agency officials, the plan that had been developed in 2001
has not been updated since 2004, when program goals were modified.
Agency officials told us the most current detailed information on
planned TEES-related efforts is the fiscal year 2008 TEES Exhibit 300,
submitted to the Office of Management and Budget (OMB), which includes
an initiative that could duplicate the Benefits Delivery Network code
conversion effort. Because VBA does not have an integrated project
management plan, it lacks:
* critical elements needed to effectively guide the initiative to
completion--such as a full description of the scope of the system
development efforts, identified stakeholders' responsibilities,
documented risks, and performance metrics; and:
* an overall approach for coordinating its various education claims
initiatives, such as its Benefits Delivery Network code conversion
effort and the development of a rules-based system to improve education
claims processing.
* Since much of the functionality for FAS has been developed, VBA could
implement a rules-based system that is integrated with FAS. However,
while the current strategy calls for using FAS as the payment system
for TEES, the existing functional requirements for an interface between
these two systems, which were developed in 2003, have not yet been
assessed and updated. Subject matter experts needed to define business
requirements for interfacing the systems have not been assigned by a
key stakeholder. Thus, it is uncertain when VBA will validate and
update functional requirements for this interface.
Conclusions:
While VBA has completed several new initiatives to provide more
automated assistance for processing education claims, the absence of an
integrated project plan for TEES could place the department at risk of
wasting millions of dollars on education claims processing initiatives
that may overlap or be duplicative. Further, without updated
requirements, VA's success in interfacing the new system with FAS, and
ultimately processing education benefits payments more efficiently,
could be jeopardized.
Recommendations:
To ensure the successful implementation of a system that provides
consistent and effective delivery of education benefits payments, we
are recommending that the Secretary of Veterans Affairs, before
approving any new funding for TEES and related systems, direct VA's
Chief Information Officer to take the following three actions:
* Develop a comprehensive, integrated TEES project plan that:
1. defines critical elements, including but not limited to project
scope, budget, schedule, risk, planning for data management, needed
knowledge and skills, and stakeholder involvement; and:
2. addresses redundancy between the requests for developing a TEES
rules-based system, the department-level code conversion effort, and
integration of the TEES development with FAS.
* Once an integrated plan among related TEES initiatives has been
completed and documented, revise the OMB Exhibit 300s for TEES and
related efforts to ensure that duplication in efforts and budgets is
eliminated and that funding is consistent with timelines.
* Obtain stakeholders' commitment of resources to ensure that
functional requirements are revalidated and updated as necessary to
provide an interface between FAS and the future VBA education systems.
Agency Comments and Our Evaluation:
We received written comments on a draft of this report from the Deputy
Secretary of Veterans Affairs. (The department's comments are
reproduced in appendix II.) In the comments, VA generally agreed with
our findings and concurred with our recommendations. The department
acknowledged that there is still much to be done to ensure that TEES is
properly planned and developed. The comments described actions planned
that respond to our recommendations, such as obtaining independent
contractor support to conduct a TEES program assessment and validation
of consolidated functional requirements in conjunction with defining
and implementing a program-level governance organization. In addition,
the department stated that its Education Project Management Service
would work with the responsible project manager to ensure TEES
timelines, requirements, risks, and budget requirements are accurately
reflected in VA's FY09 OMB Exhibit 300 submission. Further, it stated
that Education Project Management Service would coordinate with related
program offices, such as Benefits Delivery Network Rehosting and the
Finance and Accounting System, to ensure that intersections and
dependencies have been identified and accounted for in all schedules,
and eliminate the possibility of redundant efforts. If the planned
actions are properly implemented and monitored by agency leadership,
they should help to ensure that TEES will be successfully completed.
The department also provided technical comments on the draft report,
which we incorporated as appropriate.
We are sending copies of this report to the Chairman and Ranking Member
of the Committee on Veterans' Affairs. We are also sending copies to
the Secretary of Veterans Affairs and other appropriate congressional
committees. We will make copies available to other interested parties
upon request. Copies of this report will also be made available at no
charge on GAO's Web site at [hyperlink, http://www.gao.gov].
Should you or your staff have any questions about this report, please
contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. GAO staff who made major contributions to
this report are listed in appendix III.
Signed by:
Valerie C. Melvin:
Director, Human Capital and Management:
Information Systems Issues:
[End of section]
Appendix I: Veterans Affairs: Improved Planning Needed to Guide
Implementation of The Education Expert System (TEES):
Briefing to the Staff of the Subcommittee on Economic Opportunity House
Committee on Veterans' Affairs:
May 25, 2007:
Table of Contents:
Introduction:
Objectives:
Scope and Methodology:
Results in Brief:
Background:
Development Results to Date and Remaining Work:
Plans for Completing TEES:
Feasibility of Deploying TEES before FAS Conclusions:
Recommendations:
Agency Comments and Our Evaluation:
Introduction:
The Department of Veterans Affairs (VA) provides valuable benefits and
services to veterans and their families, including educational
resources to compensate for opportunities missed because of military
service. The timely provision of such services is central to the
department‘s mission.
According to the Veterans Benefits Administration (VBA),1:
the average number of days to complete original (first-time) education
claims to payment increased from 33 to 40 from fiscal year 2005 to 2006
(VBA‘s 2006 goal was 27 days) and:
the average number of days to complete supplemental education claims2
increased from 19 to 20 from 2005 to 2006 (VBA‘s 2006 goal was 13 days).
1: As reported in the department‘s Fiscal Year 2005 and Fiscal Year
2006 Performance and Accountability Reports.
2: Supplemental claims are used for subsequent school enrollment after
an original claim award or for changes in information such as address
change.
Under an initiative called The Education Expert System (TEES), VA is
taking steps to improve its education services by moving its education
claims processing systems to new technology platforms and a new
architecture.
This effort, begun in December 1999, is intended to provide faster and
more accurate processing of education claims by replacing existing
education benefit systems with a new system that will add more
automated processes, eliminate most human intervention, and work
effectively within the evolving VA corporate environment.3:
VA currently estimates that the TEES development effort will cost $77.4
million and be completed by 2011.4:
3: TEES is a follow-on to an April 1998 prototype that successfully
processes certain education claims automatically.
4: This estimate is shown in VA's 2008 OMB Exhibit 300.
Objectives:
We were asked to:
report the results of VBA's development of TEES to date, and identify
the remaining work to complete the system,
identify VBA‘s plans for completing and implementing TEES, and:
assess the feasibility of developing and deploying TEES application
processing functions before the completion of VBA‘s Finance and:
Accounting System (FAS). 5:
5: FAS is used to develop a payment record, generate various accounting
reports, and support generation and audit of benefit payments.
Scope and Methodology:
To respond to this request, we:
researched laws related to education benefits;
obtained and analyzed documentation describing the planned and ongoing
activities for TEES and FAS, including functional requirements
documents, system development and code conversion contracts for the
TEES initiative, regional office processing metrics, and cost
information;
interviewed education program officials at VBA headquarters and
interviewed project managers and other officials responsible for TEES
in VA‘s Office of Information and Technology about the various actions
taken to develop the system and costs incurred; and:
observed education claims processing and the systems currently used to
support claims processing at two of the four regional offices” Atlanta,
Georgia, and Muskogee, Oklahoma”and discussed the systems with
personnel in those offices. 6:
Our work was performed primarily at VA headquarters in Washington,
D.C., from October 2006 to May 2007 in accordance with generally
accepted government auditing standards.
6: Together these offices processed about 58 percent of total education
claims received in fiscal year 2006. Muskogee was selected because VBA
has identified Muskogee as proficient in education claims processing.
Results in Brief:
To date, VBA has enhanced education benefits claims processing by
developing certain functionalities to allow documents to be captured in
an electronic format. These enhancements include:
an automated system that allows education institutions to provide
online enrollment certification;
an automated system to allow students to provide online verification of
enrollment;
an application to allow students to phone in their monthly
verifications;
and an automated system to enable public inquiry about approved
academic programs, licensing and certification programs, and national
examinations.
However, VBA has not taken action on various other initiatives that it
identified as necessary to move its education claims processing to new
technology platforms and a new architecture and eliminate most human
intervention. These initiatives include:
moving the functionality provided by its aging, stand-alone system used
to process flight school, on the job training, and apprenticeships
claims to a platform compliant with the department‘s enterprise
architecture;
moving the processing and payment functionality used for many of the
education claims from its current antiquated system to new technology;
and:
creating a rules-based claims processing system: 7 that will process
all education benefits and training types.
7: This system will use logical rules to calculate eligibility dates
and award amounts, minimizing the need for human intervention in
processing most education claims.
VBA does not have an integrated project management plan for the TEES
initiative. According to agency officials, the plan had not been
updated since 2004 when the program goals were modified. As a result,
VBA lacks:
critical elements needed to effectively guide TEES to completion” such
as a full description of the scope of the initiative, identified
stakeholders‘ responsibilities, documented risks, and performance
metrics; and:
an overall approach for integrating its various system development and
enhancement initiatives within the department to improve education
claims processing.
VBA has developed the functionality for FAS to pay new compensation
claims, and all functionality for compensation and pension payments is
scheduled to be completed in August 2008.
While the current strategy calls for using FAS as the payment system
for TEES, the rules-based processing system has not yet been developed
and requirements for interfacing FAS with the system have not been
assessed and updated. As a result, it is unclear when VBA will be
positioned to revise the functional requirements for FAS to support
TEES.
To ensure the successful implementation of a system that provides
consistent and effective delivery of education benefits payments, we
are recommending that the Secretary of Veterans Affairs take several
actions, including directing VA‘s Chief Information Officer to develop
a comprehensive, integrated project management plan to guide the
development and implementation of TEES.
Background:
VBA Education Service:
VBA‘s Education Service administers the programs that provide
educational resources to veterans, service members, reservists, and
eligible family members of veterans.
Four regional processing offices are responsible for processing
education benefits claims:
Muskogee, Oklahoma:
Buffalo, New York:
Atlanta, Georgia:
St. Louis, Missouri:
VBA reported that in fiscal year 2006, these offices processed about
1.4 million claims, a 2 percent increase over fiscal year 2005.
Background:
Education Benefits Programs:
Beneficiaries are enrolled in one of five education benefits programs:
Chapter Number: Chapter 30: 38 USC §§ 3001,ff;
Program Name: All-Volunteer Force Educational Assistance
Program”provides education benefits to current and former active duty
personnel;
Percent of total enrollment [A]: 67;
Chapter Number: Chapter 35: 38 USC §§ 3500,ff;
Program Name: Survivors‘ and Dependents‘ Educational
Assistance”provides education benefits to survivors and dependents for
military personnel killed or totally and permanently disabled;
Percent of total enrollment [A]: 15;
Chapter Number: Chapter 1606: 10 USC §§ 16131-16137;
Program Name: Educational Assistance for Members of the Selected
Reserve”provides education benefits for members of selected reserve;
Percent of total enrollment [A]: 13;
Chapter Number: Chapter 1607: 10 USC §§ 16161-16166;
Program Name: Reserve Educational Assistance Program”provides education
benefits to National Guard and other reservists called to duty;
Percent of total enrollment [A]: 5;
Chapter Number: Chapter 32 38 USC §§ 3201,ff;
Program Name: Post-Vietnam Era Veterans‘ Educational Assistance
Program”enacted to promote the all-volunteer military program;
Percent of total enrollment [A]:
