Military Education
DOD Needs to Enhance Performance Goals and Measures to Improve Oversight of Military Academies Gao ID: GAO-03-1000 September 10, 2003Graduates of the service academies operated by the Army, Navy, and Air Force currently make up approximately 18 percent of the officer corps for the nation's armed services. The academies represent the military's most expensive source of new officers. The Department of Defense (DOD) pays the full cost of a student's 4-year education at the academies; and the related cost has increased over the past 4 years. Admission to the academies is highly competitive. The academies use a "whole person" method to make admission decisions. Recent studies by the Air Force raised questions about possible adverse effects of whole person admissions policies on student quality. GAO was asked to review all three service academies and specifically address the extent to which (1) DOD oversees the service academies, (2) applicants are granted waivers of academic standards, and (3) various groups of students differ in admissions scores and academy performance.
The Office of the Under Secretary of Defense for Personnel and Readiness (OUSD/P&R), the services, and the academies' boards of visitors conduct considerable oversight of the academies' operations and performance, but they lack a complete oversight framework. A complete oversight framework includes performance goals and measures against which the academies' performance could be better assessed. OUSD/P&R and the services use the number and type of commissioned officers as the primary measure of academy performance. OUSD/P&R requires and receives reports on academy performance from the services. While data submitted in these reports provide perspective on current performance compared with past performance, without stated performance goals and measures, these reports do not offer OUSD/P&R or the services as good an insight into the academies performance as they could. Additionally, though the academy boards of visitors serve as an external oversight mechanism to focus attention on a wide range of issues, they also do not assess the academies' performance against established performance goals and measures. The academies do not grant waivers from academic criteria or have absolute minimum scores for admission. However, under the whole person approach, the academies can admit some applicants whose academic scores are lower than might normally be competitive for admission, but who in their totality (academics, physical aptitude, and leadership) are evaluated by academy officials as being capable of succeeding at the academy. In our review of the academy classes that started in 1998 (class of 2002), we found that despite differences among various groups of students in their admissions scores and similar differences in their performance while at the academies, the differences in performance were not sizable. Some groups, such as females, performed better in some categories than the class as a whole and worse in others. Some groups (minorities, preparatory school graduates, recruited athletes, and students in the lower 30 percent of their class in terms of academic admissions scores) performed at lower levels on average in all categories than the class as a whole.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-1000, Military Education: DOD Needs to Enhance Performance Goals and Measures to Improve Oversight of Military Academies
This is the accessible text file for GAO report number GAO-03-1000
entitled 'Military Education: DOD Needs to Enhance Performance Goals
and Measures to Improve Oversight of Military Academies' which was
released on September 10, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
On January 7, 2004, this document was revised to add various footnote
references missing in the text of the body of the document, and to
enhance the formatting of Tables 15 and 16.
Report to the Subcommittee on Defense, Committee on Appropriations,
House of Representatives:
United States General Accounting Office:
GAO:
September 2003:
Military Education:
DOD Needs to Enhance Performance Goals and Measures to Improve
Oversight of Military Academies:
GAO-03-1000:
GAO Highlights:
Highlights of GAO-03-1000, a report to the Subcommittee on Defense,
Committee on Appropriations, House of Representatives
Why GAO Did This Study:
Graduates of the service academies operated by the Army, Navy, and Air
Force currently make up approximately 18 percent of the officer corps
for the nation‘s armed services. The academies represent the
military‘s most expensive source of new officers. The Department of
Defense (DOD) pays the full cost of a student‘s 4-year education at
the academies; and the related cost has increased over the past 4
years. Admission to the academies is highly competitive. The academies
use a ’whole person“ method to make admission decisions. Recent
studies by the Air Force raised questions about possible adverse
effects of whole person admissions policies on student quality. GAO
was asked to review all three service academies and specifically
address the extent to which (1) DOD oversees the service academies,
(2) applicants are granted waivers of academic standards, and (3)
various groups of students differ in admissions scores and academy
performance.
What GAO Found:
The Office of the Under Secretary of Defense for Personnel and
Readiness (OUSD/P&R), the services, and the academies‘ boards of
visitors conduct considerable oversight of the academies‘ operations
and performance, but they lack a complete oversight framework. A
complete oversight framework includes performance goals and measures
against which the academies‘ performance could be better assessed.
OUSD/P&R and the services use the number and type of commissioned
officers as the primary measure of academy performance. OUSD/P&R
requires and receives reports on academy performance from the
services. While data submitted in these reports provide perspective on
current performance compared with past performance, without stated
performance goals and measures, these reports do not offer OUSD/P&R or
the services as good an insight into the academies performance as they
could. Additionally, though the academy boards of visitors serve as an
external oversight mechanism to focus attention on a wide range of
issues, they also do not assess the academies‘ performance against
established performance goals and measures.
The academies do not grant waivers from academic criteria or have
absolute minimum scores for admission. However, under the whole person
approach, the academies can admit some applicants whose academic
scores are lower than might normally be competitive for admission, but
who in their totality (academics, physical aptitude, and leadership)
are evaluated by academy officials as being capable of succeeding at
the academy.
In our review of the academy classes that started in 1998 (class of
2002), we found that despite differences among various groups of
students in their admissions scores and similar differences in their
performance while at the academies, the differences in performance
were not sizable. Some groups, such as females, performed better in
some categories than the class as a whole and worse in others. Some
groups (minorities, preparatory school graduates, recruited athletes,
and students in the lower 30 percent of their class in terms of
academic admissions scores) performed at lower levels on average in
all categories than the class as a whole.
What GAO Recommends:
GAO recommends that the Secretary of Defense direct the Office of the
Under Secretary of Defense for Personnel and Readiness (OUSD/P&R), in
concert with the services, to further enhance performance goals and
measures to improve oversight of the operations and performance of the
service academies. In comments on a draft of this report, DOD
concurred with GAO‘s recommendation.
www.gao.gov/cgi-bin/getrpt?GAO-03-1000.
To view the full product, including the scope and methodology, click
on the link above. For more information, contact Derek Stewart at
(202) 512-5559 or stewartd@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
DOD Lacks a Complete Framework for Oversight of the Academies:
Whole Person Approach Allows Academies Flexibility to Admit Students
with a Range of Qualifications:
No Significant Differences in Admissions and Academy Performance
between Various Groups of Students:
Conclusion:
Recommendation for Executive Action:
Agency Comments:
Appendix I: Scope and Methodology:
Appendix II: Results of Statistical Analysis of Class of 2002
Admissions and Academy Performance Scores:
Admissions and Performance Scores:
Relationships between Admissions and Performance Scores:
Appendix III: Comments from the Department of Defense:
Appendix IV: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: Civilian and Military Faculty at the Service Academies:
Table 2: Academy Operating Costs and Cost Per Graduate, Fiscal Years
1999-2002.
Table 3: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U.S. Military Academy:
Table 4: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U.S. Naval Academy:
Table 5: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U.S. Air Force Academy:
Table 6: Percentage of the Selected Groups Making Up the Lower 30
percent of the Classes in Terms of Their Academic Admissions Scores, by
Academy:
Table 7: Student Performance for Selected Groups at the U.S. Military
Academy:
Table 8: Student Performance for Selected Groups at the U.S. Naval
Academy:
Table 9: Student Performance for Selected Groups at the U.S. Air Force
Academy:
Table 10: Admissions and Academy Performance Scores for the Class of
2002:
Table 11: Number of Students Graduating and Graduation Rates for the
Class of 2002:
Table 12: Admissions and Performance Scores for the Class of 2002 at
the U.S. Military Academy:
Table 13: Admissions and Performance Scores for the Class of 2002 at
the U.S. Naval Academy:
Table 14: Admissions and Performance Scores for the Class of 2002 at
the U.S. Air Force Academy:
Table 15: Regression Coefficients (Standardized Coefficients) from
Linear Regression Models Testing Correlations between Academic and
Whole Person Admissions Scores with Cumulative GPA, Cumulative MPA, and
Order of Merit for the Class of 2002 at the Service Academies:
Table 16: Regression Coefficients (Standardized Coefficients) from
Logistic Regression Models Testing Correlations Between Academic and
Whole Person Admissions Scores and the Likelihood of Graduation for the
Class of 2002 at the Service Academies:
Figures:
Figure 1: Basic Steps in the Academy Admissions Process:
Figure 2: Areas and Their Weights Considered in the U.S. Military
Academy's Whole Person Admissions Process:
Figure 3: Areas and Their Weights Considered in the U.S. Naval
Academy's Whole Person Admissions Process:
Figure 4: Areas and Their Weights Considered in the U.S. Air Force
Academy's Whole Person Admissions Process:
Figure 5: Categories of Academy Nominations:
Abbreviations:
ACT: American College Testing:
DOD: Department of Defense:
GPA: grade point average:
MPA: military performance average:
OUSD/P&R: Office of the Under Secretary of Defense for Personnel and
Readiness:
ROTC: Reserve Officer Training Corps:
SAT: Scholastic Aptitude Test:
United States General Accounting Office:
Washington, DC 20548:
September 10, 2003:
The Honorable Jerry Lewis
Chairman
The Honorable John P. Murtha
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
House of Representatives:
Graduates of the service academies operated by the Army, Navy, and Air
Force make up approximately 18 percent of the officer corps for the
nation's armed services.[Footnote 1] The academies represent the most
expensive source of new officers, compared with other sources for
officers, such as Reserve Officer Training Corps (ROTC) programs at
colleges and universities or officer candidate/training schools for
individuals who already have college degrees. The Department of Defense
(DOD) pays the full cost of providing the 4-year programs of academic
education, military training, physical conditioning, and pay for each
student.[Footnote 2] In fiscal year 2002, DOD reported costs per
graduate for the U.S. Military Academy, the U.S. Naval Academy, and the
U.S. Air Force Academy were approximately $349,000, $275,000, and
$333,000, respectively. These costs have increased over the past 4
years. To ensure the best value for the investment in the academies,
effective management principles are critical. Such principles include a
complete oversight framework, with clear roles and responsibilities, as
well as performance goals and measures against which to objectively
assess performance.
With each academy accepting about 1,200 of its more than 10,000
applicants a year, admission to the academies is highly competitive.
Applicants must be selected or obtain a nomination, such as from a
senator, representative, the President, or the Vice President, based on
the categories established by law.[Footnote 3] Most nominations are
reserved for Congress, which, therefore, has a central role in
admitting students to the academies. In addition to basic age and
medical qualifications, the academies' admissions process involves an
assessment of applicants' academic achievement (e.g., Scholastic
Aptitude Test--SAT--scores and grade point averages), physical
aptitude, and extracurricular activities (i.e., leadership potential).
Academy officials combine these assessments into a "whole person"
admissions score that is used to determine an applicant's potential to
graduate from an academy and potential fitness as a commissioned
officer. Applicants compete for admission based on these scores.
Air Force studies have raised questions about possible adverse effects
of whole person admissions policies on student quality. For example,
the Air Force found that its whole person assessments and resulting
admissions scores have led the Air Force Academy to admit an increasing
number of students whose academic qualifications are below academic
minimums, as well as to admit an increasing number of students
recruited largely to participate in varsity intercollegiate athletics.
The House of Representatives report on defense appropriations for
fiscal year 2003[Footnote 4] directed that we perform reviews of all
three service academies and their respective preparatory
schools.[Footnote 5] As part of the review of the service academies, we
were also directed to obtain student and faculty perceptions of various
aspects of student life at the academies. Issues associated with the
academy preparatory schools and the results of surveys on aspects of
student life are addressed in separate reports.[Footnote 6]
As agreed with your offices, this report addresses the following
questions, to what extent:
does DOD oversee the academies' operations and performance?
are applicants granted waivers from academic criteria for admissions?
do various groups of students differ in admissions scores and academy
performance?
In addition to reviewing documents and interviewing officials at all
three academies, the service headquarters, the Office of the Under
Secretary of Defense for Personnel and Readiness (OUSD/P&R), and the
academies' boards of visitors, we reviewed admissions policies and
procedures and observed their use by academy officials in evaluating
applications being considered for the incoming class of 2007. We also
obtained and analyzed admissions and performance data for the student
class that graduated in 2002. To compare student admissions
qualifications and performance at the academies, we identified six
major groups of students common to all academies: females, minorities,
academy preparatory school graduates, recruited athletes, prior
enlisted personnel, and students whose academic admission scores fell
in the lower 30 percent of the entering class.[Footnote 7] Data on
student performance included academic grade point average; military
performance average, which is similar to a performance evaluation for
commissioned officers; and class rank.[Footnote 8] It also included
graduation rate. Other issues, such as recent controversies associated
with alleged sexual assault, did not fall within the scope of this
review. Further details on our scope and methodology are in appendix I.
We conducted our work from October 2002 through May 2003 in accordance
with generally accepted government auditing standards.
Results in Brief:
The Office of the Under Secretary of Defense for Personnel and
Readiness (OUSD/P&R), the services, and the academies' boards of
visitors conduct considerable oversight of the academies' operations
and performance, but they lack a complete oversight framework. In 1991,
our report concluded that better oversight of the academies was needed
and made recommendations to improve DOD oversight. Since then, DOD has
taken measures to address these issues, including establishing guidance
on oversight of the academies and uniform academy cost reporting.
However, DOD has not established a complete oversight framework, which
would include not only clear roles and responsibilities, but also
performance goals and measures against which to objectively assess
performance. OUSD/P&R, the services, and the academies' boards of
visitors have different oversight roles, but largely conduct oversight
activities without the benefit of formalized performance goals and
measures. OUSD/P&R and the services use the number and types of
commissioned officers as the primary measure of academy performance.
OUSD/P&R requires and receives reports on academy performance from the
services.
While data submitted in these reports provide perspective on such
performance measures as graduation rates, admissions trends for women
and minorities, and information on the quality of admitted students,
without stated performance goals and measures, these data do not offer
OUSD/P&R or the services as good an insight into the academies'
performance as they could. For example, the data collected by the
academies show that the graduation rates have increased in the last 10
years; however, there is no stated goal for graduation rate against
which to judge whether this rate of increase is adequate. Other data
collected by the academies indicate that the percentage of females and
minorities has fluctuated over the last 3 years, but apart from
admissions targets used by the Military Academy, there are no stated
goals against which to measure the adequacy of these admissions trends.
Additionally, academy officials regularly analyze data on student
performance to determine the extent to which admissions standards can
be changed to improve overall student performance at the academies.
However, there are no stated goals for student body performance, apart
from minimum graduation standards such as the cumulative academic grade
point average, that might help the academies and other oversight bodies
assess overall student performance. Additionally, each academy's board
of visitors--an external oversight mechanism--focuses attention and
actions on a wide range of operational and quality of life issues at
the academies. However, the boards do not evaluate academy performance
against established performance goals and measures. Without formal
goals and measures that are, moreover, linked to mission statements,
oversight bodies do not have sufficient focus for their efforts and
cannot systematically assess an organization's strengths and weaknesses
nor identify appropriate remedies that would help them achieve the best
value for the nation's investment in the academies.
The academies do not grant waivers from academic criteria or have
absolute minimum scores for admission. Under the whole person approach,
the academies can admit some applicants whose academic scores are lower
than might normally be competitive for admission, but who in their
totality (academics, physical aptitude, and leadership) are evaluated
by academy officials as being qualified and capable of succeeding at
the academy. The only admissions criteria with an absolute minimum
score for qualifying for admissions is physical aptitude. The academic
and leadership criteria have a range of qualifying scores based on what
general levels of ability are considered competitive during the
admissions process. If an applicant's score is lower than the
competitive range in academics, then admission officials have some
flexibility in (1) further considering the applicant by re-examining
the student's record for information that can produce further insight
about his or her academic achievement and (2) weighing the extent to
which the leadership component of the whole person score may offset the
low component. It is possible for students to be admitted whose
academic scores were not as competitive as some of their peers who may
not have been admitted. The applicant is considered a risk and is
evaluated through a deliberative process by academy officials on the
basis of their judgment of whether the applicant is fully qualified and
capable of succeeding at that academy. The subjective nature of this
approach is consistent with the intent of the whole person concept, by
which the academies want to admit students who also demonstrate
leadership characteristics that cannot be quantified by purely
objective scoring methods. Academy officials do not consider this
approach to represent an academic waiver, but instead their judicious
assessment of the whole person.
In our review of the academy classes that started in 1998 (class of
2002), we found differences among various groups of students in their
admissions scores and similar differences in their performance while at
the academies; the differences in performance were not sizable. For the
class data we reviewed, minorities, academy preparatory school
graduates, recruited athletes, and prior enlisted students[Footnote 9]
all had lower average admissions scores than the average for the class
as a whole. Of those students in the lower 30 percent of the class in
terms of academic admissions scores, about 44 percent were recruited
athletes, between 25 and 31 percent were minorities, and between 20 and
34 percent were preparatory school graduates. Regarding performance, we
found differences at the academies between selected groups (i.e.,
females, minorities, preparatory school graduates, recruited athletes,
prior enlisted students, and students in the lower 30 percent of the
class in terms of academic admissions scores) and the class as a whole.
Those differences varied but were generally not sizable. For example,
females at one academy had a lower graduation rate than the class as a
whole but a higher average academic grade point average and a higher
average class rank. Some groups at all academies--such as minorities,
preparatory school graduates, recruited athletes, and students in the
lower 30 percent of their class in terms of academic admissions scores-
-performed at lower levels on average in all categories than the class
as a whole, but these differences were not significant. For example,
one of the lowest average academic grade point averages among the
groups we reviewed was 2.61, whereas the average for the class as a
whole at that academy was 2.93. A 2.0 grade point average is required
to graduate. The lowest graduation rate for the class we reviewed was
65 percent for the students in the lower 30 percent of their class in
terms of academic admissions scores at one academy. The average
graduation rate for the class as a whole at that academy was 74
percent.
We are making a recommendation to improve DOD's oversight of operations
and performance at the academies through the enhancement of performance
goals and measures. In comments on a draft of this report, DOD
concurred with our recommendation.
Background:
The Departments of the Army, the Navy, and the Air Force each have
their own educational institutions (academies) to produce a portion of
each branch's officer corps:[Footnote 10]
* U.S. Military Academy (West Point, N.Y.), established in 1802;
* U.S. Naval Academy (Annapolis, Md.), established in 1845; and:
* U.S. Air Force Academy (Colorado Springs, Colo.), established in
1954.
The academies are structured to provide a curriculum critical to the
development of successful future officers in academic, military, and
physical areas of achievement. Additionally, the academies emphasize
the moral and ethical development of students through their respective
honor codes and concepts.
There are approximately 4,000 students enrolled at each of the three
service academies at any given time, each comprising four classes. In
December 2002, Congress authorized an annual increase of up to 100
students until the total number reaches 4,400 for each
academy.[Footnote 11] In 2002 the Military Academy graduated 968
students; the Naval Academy 977 students; and the Air Force Academy 894
students. Faculty at the U.S. Military Academy and the U.S. Air Force
Academy are comprised predominantly of military officers (79 and 75
percent, respectively), while at the U.S. Naval Academy 59 percent of
the faculty are civilians. Table 1 shows the composition of the faculty
at the service academies.
Table 1: Civilian and Military Faculty at the Service Academies:
Service academy: U.S. Military Academy; Total number of faculty: 622;
Total number of civilian faculty (% of faculty): 131 (21%); Total
number of military faculty (% of faculty): 491 (79%).
Service academy: U.S. Naval Academy; Total number of faculty: 555;
Total number of civilian faculty (% of faculty): 326 (59%); Total
number of military faculty (% of faculty): 229 (41%).
Service academy: U.S. Air Force Academy; Total number of faculty: 490;
Total number of civilian faculty (% of faculty): 123 (25%); Total
number of military faculty (% of faculty): 367 (75%).
Service academy: Total; Total number of faculty: 1,667; Total number of
civilian faculty (% of faculty): 580 (35%); Total number of military
faculty (% of faculty): 1,087 (65%).
Source: DOD.
Note: Faculty information is based on a snapshot of each academy in
February 2003.
[End of table]
DOD reports that the total cost to operate all three academies in
fiscal year 2002 was $990.7 million. Table 2 shows the reported
operating costs and cost per graduate for each academy from fiscal year
1999 through fiscal year 2002. We did not independently verify these
costs.
Table 2: Academy Operating Costs and Cost Per Graduate, Fiscal Years
1999-2002:
Academy: Military Academy; Cost Category: Total operating costs; Fiscal
year 1999: $301,058,452; Fiscal year 2000: $330,603,820; Fiscal year
2001: $336,416,716; Fiscal year 2002: $364,971,975.
Cost Category: Cost per graduate; Fiscal year 1999: 312,150; Fiscal
year 2000: 320,120; Fiscal year 2001: 339,318; Fiscal year 2002:
349,327.
Academy: Naval Academy; Cost Category: Total operating costs; Fiscal
year 1999: 245,749,679; Fiscal year 2000: 253,817,467; Fiscal year
2001: 273,809,865; Fiscal year 2002: 292,696,358.
Cost Category: Cost per graduate; Fiscal year 1999: 254,983; Fiscal
year 2000: 256,931; Fiscal year 2001: 266,033; Fiscal year 2002:
275,001.
Academy: Air Force Academy; Cost Category: Total operating costs;
Fiscal year 1999: 277,639,005; Fiscal year 2000: 314,972,559; Fiscal
year 2001: 321,335,152; Fiscal year 2002: 333,056,023.
Cost Category: Cost per graduate; Fiscal year 1999: 305,945; Fiscal
year 2000: 305,133; Fiscal year 2001: 313,456; Fiscal year 2002:
322,750.
Source: DOD.
[End of table]
Prospective students must meet basic eligibility requirements for
appointment to an academy. They must (1) be unmarried, (2) be a U.S.
citizen, (3) be at least 17 years of age and must not have passed their
twenty-third birthday on July 1 of the year they enter an academy, (4)
have no dependents, and (5) be of good moral character.[Footnote 12]
After determining eligibility, a candidate submits an application to a
preferred academy or academies. Each submitted application is required
to include information such as, but not limited to, the candidate's (1)
SAT scores (or American College Testing--ACT--examination scores); (2)
high school grade point average (and class rank, if possible); (3)
physical aptitude scores; (4) medical examination results; and (5)
extracurricular activities. The academies admit those candidates that
have secured a nomination and who represent, in the opinion of academy
officials, the best mixture of attributes (academic, physical, and
leadership) necessary to ensure success at the academies and as
military officers.
The military academies use a "whole person" method to assess potential
candidates in three major areas: (1) academics, (2) physical aptitude,
and (3) leadership potential. Each academy uses the same basic
approach. Admissions assessments are weighted toward academic scores
that include objective tests and high school performance. Leadership
potential is measured by assessing athletic and non-athletic
extracurricular activities. Subjective assessments of potential
candidates in these major areas also contribute to final admissions
"scores." Such assessments include interviews with prospective
candidates, teacher/coach evaluations, and analyses of writing samples.
Though medical criteria differ between services, the medical
examinations are conducted according to the same standards, under a
joint DOD Medical Examination Review Board that manages the medical
examination process and records for applicants to all
academies.[Footnote 13]
Each academy is authorized to permit up to 60 foreign students to
attend at any given time on a reimbursable basis by their country of
origin.[Footnote 14] This number does not count against the authorized
student strength of the academies. The admission of foreign students is
covered by separate policies and procedures. Our review was limited to
the policies and procedures for admitting U.S. citizens to the
academies. Figure 1 shows the basic steps in the admissions process for
all U.S. applicants.
Figure 1: Basic Steps in the Academy Admissions Process:
[See PDF for image]
[End of figure]
Students who are disenrolled from an academy after the start of their
third year may be required to complete a period of active duty enlisted
service of up to 4 years or may be required to reimburse the federal
government for the cost of their education. Those who are disenrolled
in their first 2 years do not incur an active service or reimbursement
obligation.[Footnote 15]
United States Military Academy Admissions Process:
The United States Military Academy's admissions evaluation considers
academics, leadership, and physical aptitude. Academic considerations
include above-average high school or college academic records as well
as strong performance on SAT/ACT. Additionally, the Military Academy
considers recommendations from English, mathematics, and science
teachers. The leadership potential considers demonstrations of
leadership and initiative in sports, school, community, or church
activities and strong recommendations from faculty and community
leadership and is a more subjective assessment of character. Physical
aptitude is based on a scored standardized test. This test is made up
of pull-ups for men or the flexed-arm hang for women, push-ups,
standing long jump, basketball throw, and shuttle run. Figure 2 shows
the areas considered and the weights assigned to each area in the U.S.
Military Academy's whole person admissions process.
Figure 2: Areas and Their Weights Considered in the U.S. Military
Academy's Whole Person Admissions Process:
[See PDF for image]
[End of figure]
United States Naval Academy Admissions Process:
The United States Naval Academy's admissions evaluation considers
academics, leadership, physical aptitude, and technical interest.
Academic considerations include above-average high school or college
academic records as well as strong performance on SAT/ACT.
Additionally, the Naval Academy considers recommendations from English
and mathematics teachers. Assessment of leadership potential represents
a subjective evaluation of character in which the academy considers
demonstrations of leadership in terms of extracurricular activities in
sports, school, community, or church and strong recommendations from
faculty and community leadership. Physical aptitude is based on a
scored, standardized test consisting of pull-ups for men or the flexed-
arm hang for women, push-ups, standing long jump, basketball throw, and
shuttle run. Additionally, the Naval Academy considers the technical
interest of a prospective student, which is measured through a
questionnaire in the application packet and used to gauge interest in
pursuing a technical degree. The intent of this requirement is to admit
students that are interested in pursuing technical degrees,
specifically nuclear and maritime engineering. The admissions board can
also apply further points to an applicant's overall whole person score
based on further consideration of an applicant's record, including such
things as the results of the evaluation form filled out by the Naval
Academy representative who interviewed the applicant. Figure 3 shows
the areas considered and the weights assigned to each area in the U.S.
Naval Academy's whole person admissions process.
Figure 3: Areas and Their Weights Considered in the U.S. Naval
Academy's Whole Person Admissions Process:
[See PDF for image]
[End of figure]
United States Air Force Academy Admissions Process:
The United States Air Force Academy's admissions evaluation considers
academics, leadership, and an assessment by the selections panel.
Academic considerations include above-average high school or college
academic records as well as strong performance on SAT/ACT.
Additionally, the Air Force Academy considers recommendations from
English and mathematics teachers. Under leadership potential, the
academy considers extracurricular activities in sports, school,
community, or church and strong recommendations from faculty and
community leadership. Finally, the Air Force Academy Selections Panel
makes an assessment of all potential students. This assessment is
composed of a pass/fail score from the physical aptitude examination
and the evaluation of the academy's liaison officer evaluation, made
after interviewing the applicant. The physical aptitude examination is
made up of pull-ups for men or the flexed-arm hang for women, push-ups,
standing long jump, basketball throw, and shuttle run. The leadership
potential area and the admissions board include the more subjective
assessments of a potential student. Figure 4 shows the areas considered
and the weights assigned to each area in the U.S. Air Force Academy's
whole person admissions process.
Figure 4: Areas and Their Weights Considered in the U.S. Air Force
Academy's Whole Person Admissions Process:
[See PDF for image]
[End of figure]
Nomination and Appointment of Candidates:
The President of the United States alone appoints candidates to the
academies.[Footnote 16] Before receiving an appointment, all candidates
must secure one or more nominations according to the following
categories:[Footnote 17]
* congressional (including a U.S. senator, representative, delegate, or
the Vice President);
* service-connected (including, among others, children of disabled
veterans, enlisted personnel in the active or reserve components, and
students from ROTC programs or other designated honor school
graduates); and:
* other (including the academy superintendents' nominees and other
nominees to bring the incoming class to full strength).
Figure 5 shows the approximate distribution of categories of academy
nominations, based on the types and numbers of nominees per category
allowed by law.
Figure 5: Categories of Academy Nominations:
[See PDF for image]
Note: Percentages do not add to 100 due to rounding.
[End of figure]
Oversight of the Academies:
Oversight of the academies is the responsibility of three principal
organizations: OUSD/P&R, the service headquarters, and the board of
visitors of each academy. According to Department of Defense Directive
1322.22 (Service Academies),[Footnote 18] OUSD/P&R serves as the DOD
focal point for matters affecting the academies and has responsibility
to assess academy operations and establish policy and guidance for
uniform oversight and management of the military academies. The
military departments perform the primary DOD oversight function for
their respective academies. The superintendent of each academy reports
directly to the uniformed head of his respective service (the Chiefs of
Staff for the Army and the Air Force and the Chief of Naval Operations
for the Navy), in accordance with the chain of command for each
service. Each academy also has a board of visitors, mandated by
law,[Footnote 19] that is comprised of congressional members and
presidential appointees. These boards focus attention and action on a
wide range of operational and quality of life issues at the academies.
As educational institutions, the service academies are also overseen by
several nongovernmental organizations that are outside DOD purview.
Each academy undergoes periodic review by a higher-education
accreditation body associated with its region of the country,[Footnote
20] usually involving a full review every 10 years with an interim
review every 5 years. The accreditation bodies review such areas as
core curriculum, strategic planning, self-assessments, diversity of
faculty and students, and faculty credentials. The athletic programs of
the academies are also subject to periodic certification by the
National Collegiate Athletic Association. This body reviews academy
athletics in terms of such issues as finances and impact on the
education mission of the academies. We limited our review of oversight
of the academies to DOD organizations and the boards of visitors.
DOD Lacks a Complete Framework for Oversight of the Academies:
The OUSD/P&R, the services, and the academies' boards of visitors
conduct many oversight activities, but they lack a complete oversight
framework. A complete oversight framework includes not only clear roles
and responsibilities, but also performance goals and measures against
which to objectively assess performance. Such elements embody the
principles of effective management in which achievements are tracked in
comparison with plans, goals, and objectives and the differences
between actual performance and planned results are analyzed. Without
formal goals and measures, oversight bodies do not have sufficient
focus for their efforts and cannot systematically assess an
organization's strengths and weaknesses nor identify appropriate
remedies that would permit DOD to achieve the best value for the
investment in the academies. In a prior report,[Footnote 21] GAO
concluded that better external oversight of the academies was needed to
provide useful guidance and suggestions for improvement. The report
recommended that DOD improve oversight of the academies through such
measures as establishing a focal point for monitoring academy issues in
the Office of the Secretary of Defense and establishing guidance on
uniform cost reporting.
OUSD/P&R and the services have established clear roles and
responsibilities for oversight of the academies, with the former
serving as the focal point for issues affecting all academies and the
latter having direct oversight authority over their respective
academies. DOD established guidance in 1994 for the oversight of the
academies[Footnote 22] and for uniform reporting of costs and
resources.[Footnote 23] OUSD/P&R is directly involved in those policy
issues that affect all academies and require DOD-level attention and
legislative matters. For example, the office was recently the DOD focal
point on the issue of increasing authorized enrollment at the academies
from 4,000 to 4,400. With respect to the academies, the office is
chiefly concerned with monitoring the degree to which the services are
meeting their goals for the accession of new officers.[Footnote 24] The
office also coordinates major studies that affect the academies, such
as a November 1999 report on the career progression of minority and
women officers.
The services are responsible for direct oversight of their respective
academies; and the academies are treated similarly to major military
commands. The superintendents of the academies are general/flag
officers who report directly to the uniformed heads of their services
(the Chiefs of Staff for the Army and the Air Force and the Chief of
Naval Operations for the Navy). In addition to overseeing the
academies' budget through the same approval process as a major command
activity, the services oversee the academies' operations and
performance primarily through the academies' goal of meeting service
officer accession targets. The superintendents are responsible for
meeting those targets and, in so doing, are given wide discretion in
such areas as modifying their specific admissions objectives and the
process for matching graduates with service assignments. The service
headquarters use a number of mechanisms to oversee academy performance.
For example, each service headquarters provides officer accession
targets to the academies so that the assignment of graduates and the
make up of incoming student classes can be modified as necessary. In
addition to general numbers of officers, each service also has a number
of specialty officer fields that need to be filled, and the services
also monitor the extent to which the academies will be able to meet
those accession goals.
The services also directly oversee the academies by requiring the
superintendents to report on and discuss their operations. For example,
the Air Force uses an annual forum of the most senior Air Force
officers to focus on the Air Force Academy with respect to how it is
meeting the needs of the operational Air Force. The Navy uses similar
senior officer conferences and frequent interaction between the
superintendent and Navy headquarters to conduct oversight. The Army
uses the U.S. Military Academy Forum, comprised of senior Army
officers, to address academy operations issues. The superintendents of
the three academies also hold annual meetings to discuss issues common
to all academies. These mechanisms have resulted in such academy
actions as curriculum changes to increase the number of technical
degree majors, increasing language requirements, and increasing the
number of students attending the academies.
While OUSD/P&R and the services conduct a wide variety of oversight
activity, there are few stated performance goals against which to
measure academy operations and performance. Each of the academies has a
strategic plan that is focused on providing quality military and
professional training and education in order to commission highly
capable junior officers. These plans are approved by the service
headquarters but are not generally used by the services as benchmarks
against which to measure academy performance, and they do not contain
specific goals against which to measure student performance. OUSD/P&R
is required to assess and monitor academy operations based on the
information provided in annual reports it requires from the service
secretaries.[Footnote 25] These reports provide data on various aspects
of performance, such as student demographics and trends, student
quality, admissions and attrition trends, compensation for students and
faculty, leadership and honor systems, and incidents of indiscipline.
The reports provide OUSD/P&R and the services with information on
current and past performance for academy operations, but apart from
officer accession goals, neither OUSD/P&R nor the services have
specific stated performance goals against which to compare the
information provided in the assessment reports, thus they do not have
an explicit basis for judging the adequacy of their performance. For
example, the data collected by the academies show that graduation rates
have increased in the last 10 years; however, there is no stated goal
for a graduation rate against which to judge whether this rate of
increase is adequate. Other data collected by the academies indicate
that the percentage of females and minorities has fluctuated over the
last 3 years, but apart from admissions targets used by the U.S.
Military Academy, there are no stated goals against which to assess
these trends. Additionally, academy officials regularly analyze data on
student body performance to determine the extent to which admissions
standards can be changed to affect student body performance. However,
there are no stated goals for student body performance, apart from
minimum graduation standards, that might help the academies and other
oversight bodies assess overall student performance.
The oversight efforts of each academy's board of visitors are similarly
limited by the absence of sufficient performance goals and measures.
Each of the academies has a board of visitors, mandated by law[Footnote
26] and comprised of Members of Congress and presidential appointees,
that is outside the DOD chain of command. The boards have a broad legal
mandate to inquire into all aspects of academy operations.[Footnote 27]
The boards meet several times a year to be briefed on and discuss
academy operations and must conduct an annual visit to their respective
academies. During these visits, the boards are briefed by academy staff
on such issues as admissions, curriculum, recruiting, athletics, morale
and welfare, and construction programs; they also interview students to
obtain their perceptions of life at the academies. The boards also
address inquiries to academy staff, which are usually followed up at
subsequent meetings, and they make suggestions to improve operations or
quality of life at the academies. For example, boards of visitors have
recommended increased recruiting of qualified minority applicants from
various congressional districts and increased surveying of students on
quality of life issues.
The boards submit annual reports to the President on the status of and
issues at the academies but do not evaluate academy operations and
performance against established performance goals. The boards of
visitors do not have dedicated staffs to conduct their work, and though
board members may inquire into any aspect of academy operations, the
agenda is set largely by the briefings presented to the boards by
academy officials. Academy officials with whom we spoke were generally
satisfied with the oversight provided by the boards of visitors, though
there were concerns at the Air Force Academy about poor attendance by
board members during annual visits to the academy.
Whole Person Approach Allows Academies Flexibility to Admit Students
with a Range of Qualifications:
The academies do not grant waivers from academic criteria but do not
have absolute minimum scores for admission. Under the whole person
approach, the academies can admit some applicants whose academic scores
are lower than might normally be competitive for admission, but who in
their totality (academics, physical aptitude, and leadership potential)
are deemed an acceptable risk and qualified to attend an academy. This
admissions approach is consistent with the intent of the academies to
admit students who also demonstrate leadership and initiative
characteristics, which cannot be quantified by purely objective scoring
methods.
When conducting their admissions processes, the academies do not set
absolute minimum scores for academic ability. Rather, they establish a
range of scores that would be considered competitive, based on past
incoming class performance and academy research on the overall quality
of the applicant pool. Prior to 2002, the Air Force Academy set
absolute minimum academic scores, and a waiver was required to further
consider an applicant who fell below that minimum, no matter how high
his or her scores in the leadership area. However, the Air Force
Academy no longer has absolute minimums and uses the same competitive
range approach as the other academies. Under this approach, if an
applicant's academic score is lower than the competitive range
guidelines, academy officials have some flexibility to further consider
the applicant. Academy officials will re-examine the applicant's record
for information that might provide further insight about his or her
academic achievement. For example, officials may contact high school
teachers to inquire about the types and difficulty of the classes the
applicant has been taking and his or her performance in those classes.
Academy officials will also weigh the extent to which the leadership
component of the applicant's whole person score offset the low
component. The applicant is considered a risk and is evaluated through
a deliberative process by academy officials on the basis of their
judgment of whether the applicant is fully qualified and capable of
succeeding at that academy. The subjective nature of this approach is
consistent with the intent of the whole person concept, by which the
academies want to admit students who also demonstrate leadership
characteristics that cannot be quantified by purely objective scoring
methods. Academy officials do not consider these judgments to
constitute a waiver of academic standards, but rather a judicious
assessment of the whole person. The process for assessing those
applicants whose academic scores are lower than might normally be
competitive is nonetheless similar to the former Air Force Academy
process for granting waivers.
With over 10,000 applicants[Footnote 28] for each academy each year and
about 1,200 students admitted, the academic standards are high. Academy
data show that the academic quality of the applicants has remained high
over the past 4 years, and the competitive ranges for academic scores
used by the academies have remained the same or have increased during
this time. However, it is possible for students to be admitted whose
academic scores were not as competitive as some other applicants who
may not have been admitted. Senators, representatives, and delegates
may submit up to 10 nominees for each student vacancy available to him
or her per academy. They may choose to designate one as a principal
nominee.[Footnote 29] If an applicant receives a principal nomination
and is in all other respects qualified, the academies must admit that
applicant, even over an applicant on the same senator's, delegate's, or
representative's nomination list with higher academic and/or whole
person scores. The other nominated names become alternates for possible
admission later in the admissions process.
Though some academies award credit for the extent to which an applicant
surpasses the standards of the physical aptitude examination, there are
minimum standards for the physical test that must be met. None of the
academies uses a system of "waivers," except for medical conditions. An
applicant can be waived for a medical condition, based on the
deliberation and judgment of DOD medical personnel and the academy
superintendent. For example, an applicant who is disqualified due to a
vision condition may apply for and receive a waiver, based on
subsequent surgical vision correction or determination by the academy
superintendent that the applicant would be able to serve on active duty
without the vision condition being a problem.
No Significant Differences in Admissions and Academy Performance
between Various Groups of Students:
In our review of the academy classes that started in 1998 (class of
2002), we found differences among various groups of students in their
admissions scores and similar differences in their performance while at
the academies, but the differences were not significant in magnitude.
In terms of performance after admission to the academies,[Footnote 30]
differences between these student groups and the class as a whole were
also not sizable. We reviewed data for the following distinct
groups:[Footnote 31]
* overall class,
* females,
* minorities,
* academy preparatory school graduates,
* recruited athletes,
* prior enlisted, and:
* lower 30 percent of class by academic admissions scores.
For the class data we reviewed, minorities, academy preparatory school
graduates, recruited athletes, and prior enlisted students all had
lower average admissions scores than the average for the class as a
whole, though these differences varied. The differences between groups
and the class as a whole were not sizable, generally falling within 5
percent. Those differences that were statistically significant and
outside the 5 percent range were still generally less than 10 percent
of the class as a whole. Tables 3, 4, and 5, show the average
admissions scores for the selected groups in the class that started in
1998 at the Military, Naval, and Air Force Academies, respectively.
Although each academy uses the same fundamental whole person approach,
they use different scales to calculate scores. Therefore, the academic
and whole person scores cannot be compared across academies.
Table 3: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U.S. Military Academy:
Average admissions score: Academic score; Overall (1,246): 600; Females
(192): 603; Minorities (269): 583; Preparatory school graduates (184):
546[B]; Recruited athletes (279): 558[A]; Prior enlisted (31): 594;
Lower 30 percent of admissions class (377): 532[B].
Average admissions score: Whole person score; Overall (1,246): 6,006;
Females (192): 6,022; Minorities (269): 5,865; Preparatory school
graduates (184): 5,645[A]; Recruited athletes (279): 5,814; Prior
enlisted (31): 5,861; Lower 30 percent of admissions class (377):
5,609[A].
Source: GAO analysis, from Military Academy sources.
[A] Denotes a group average or percentage with a statistically
significant difference greater than 5% from the overall average or
percentage.
[B] Denotes a group average or percentage with a statistically
significant difference greater than 10% from the overall average or
percentage.
[End of table]
Table 4: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U.S. Naval Academy:
Average admissions score: Academic score; Overall (1,226): 618; Females
(190): 624; Minorities (221): 594; Preparatory school graduates (146):
545[B]; Recruited athletes (380): 596; Prior enlisted (76): 570[A];
Lower 30 percent of admissions class (368): 544[B].
Average admissions score: Whole person Score; Overall (1,226): 65,732;
Females (190): 65,719; Minorities (221): 63,769; Preparatory school
graduates (146): 61,254[A]; Recruited athletes (380): 64,233; Prior
enlisted (76): 62,256[A]; Lower 30 percent of admissions class (368):
61,404[A].
Source: GAO analysis, from Naval Academy sources.
[A] Denotes a group average or percentage with a statistically
significant difference greater than 5% from the overall average or
percentage.
[B] Denotes a group average or percentage with a statistically
significant difference greater than 10% from the overall average or
percentage.
[End of table]
Table 5: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U.S. Air Force Academy:
Average admissions score: Academic Score; Overall (1,216): 3,202;
Females (190): 3,216; Minorities (229): 3,123; Preparatory school
graduates (157): 3,112; Recruited athletes (312): 3,043; Prior enlisted
(44): 3,188; Lower 30 percent of admissions class (366): 2,863[B].
Average admissions score: Whole person Score; Overall (1,216): 798;
Females (190): 805; Minorities (229): 782; Preparatory school graduates
(157): 774; Recruited athletes (312): 773; Prior enlisted (44): 792;
Lower 30 percent of admissions class (366): 751[A].
Source: GAO analysis, from Air Force Academy sources.
[A] Denotes a group average or percentage with a statistically
significant difference greater than 5% from the overall average or
percentage.
[B] Denotes a group average or percentage with a statistically
significant difference greater than 10% from the overall average or
percentage.
[End of table]
Of those students in the lower 30 percent of the class in terms of
academic admissions scores, about 44 percent were recruited athletes,
between 25 and 31 percent were minorities, and between 20 and 34
percent were preparatory school graduates. Table 6 shows the percentage
of the selected groups making up the lower 30 percent of the classes in
terms of their academic admissions scores, by academy.
Table 6: Percentage of the Selected Groups Making Up the Lower 30
percent of the Classes in Terms of Their Academic Admissions Scores, by
Academy:
Numbers in percent.
Military Academy; Females: 16; Minorities: 31; Preparatory school
graduates: 34; Recruited athletes: 44; Prior enlisted: 3.
Naval Academy; Females: 13; Minorities: 29; Preparatory school
graduates: 32; Recruited athletes: 45; Prior enlisted: 13.
Air Force Academy; Females: 14; Minorities: 25; Preparatory school
graduates: 20; Recruited athletes: 44; Prior enlisted: 5.
Source: GAO analysis, from DOD sources.
Note: Numbers do not add to 100 percent because each of the groups can
contain members of another group.
[End of table]
We also found differences in performance after admission to the
academies between selected groups and the class as a whole. For
example, females at the Naval Academy had a lower graduation rate than
the class as a whole, but they had a higher average academic grade
point average (cumulative GPA) than the class as a whole and higher
average class rank (order of merit). The differences in performance
between the selected groups and the class as a whole were not sizable,
generally falling within 5 percent. Those differences that were
statistically significant and outside the 5 percent range were still
generally less than 10 percent of the class as a whole. Tables 7, 8,
and 9 show how the selected groups performed at the Military, Naval,
and Air Force Academies, respectively. See appendix II for further
information on comparisons of performance by defined student groups.
Table 7: Student Performance for Selected Groups at the U.S. Military
Academy:
Performance score: Average cumulative GPA; Overall (1,246): 2.99;
Females (192): 2.99; Minorities (269): 2.82; Preparatory school
graduates (184): 2.61[A]; Recruited athletes (279): 2.81; Prior
enlisted (31): 3.14; Lower 30 percent of admissions class (377):
2.66[A].
Performance score: Average cumulative MPA; Overall (1,246): 3.28;
Females (192): 3.26; Minorities (269): 3.21; Preparatory school
graduates (184): 3.26; Recruited athletes (279): 3.20; Prior enlisted
(31): 3.37; Lower 30 percent of admissions class (377): 3.21.
Performance score: Average order of merit; Overall (1,246): 3.03;
Females (192): 3.04; Minorities (269): 2.86[A]; Preparatory school
graduates (184): 2.75[A]; Recruited athletes (279): 2.90; Prior
enlisted (31): 3.06; Lower 30 percent of admissions class (377):
2.78[A].
Performance score: Graduation rate; Overall (1,246): 78%; Females
(192): 76%; Minorities (269): 71%[A]; Preparatory school graduates
(184): 72%; Recruited athletes (279): 76%; Prior enlisted (31): 71%;
Lower 30 percent of admissions class (377): 71%[A].
Source: GAO analysis, from Military Academy sources.
[A] Denotes a group average or percentage with a statistically
significant difference greater than 5% from the overall average or
percentage.
[B] Denotes a group average or percentage with a statistically
significant difference greater than 10% from the overall average or
percentage.
[End of table]
Table 8: Student Performance for Selected Groups at the U.S. Naval
Academy:
(Continued From Previous Page)
Performance score: Average cumulative GPA; Overall (1,226): 2.97;
Females (190): 3.01; Minorities (221): 2.82[A]; Preparatory school
graduates (146): 2.67[A]; Recruited athletes (380): 2.86; Prior
enlisted (76): 3.02; Lower 30 percent of admissions class (368):
2.67[A].
Performance score: Average cumulative MPA; Overall (1,226): 3.12;
Females (190): 3.16; Minorities (221): 3.02; Preparatory school
graduates (146): 2.99; Recruited athletes (380): 3.08; Prior enlisted
(76): 3.19; Lower 30 percent of admissions class (368): 3.00.
Performance score: Average order of merit; Overall (1,226): 489;
Females (190): 456[A]; Minorities (221): 590[B]; Preparatory school
graduates (146): 658[B]; Recruited athletes (380): 551[B]; Prior
enlisted (76): 453; Lower 30 percent of admissions class (368): 661[B].
Performance score: Graduation rate; Overall (1,226): 80%; Females
(190): 71%[B]; Minorities (221): 75%; Preparatory school graduates
(146): 77%; Recruited athletes (380): 79%; Prior enlisted (76): 72%;
Lower 30 percent of admissions class (368): 76%[A].
Source: GAO analysis, from Naval Academy sources.
[A] Denotes a group average or percentage with a statistically
significant difference greater than 5% from the overall average or
percentage.
[B] Denotes a group average or percentage with a statistically
significant difference greater than 10% from the overall average or
percentage.
[End of table]
Table 9: Student Performance for Selected Groups at the U.S. Air Force
Academy:
Performance score: Average cumulative GPA; Overall (1,216): 2.93;
Females (190): 2.97; Minorities (229): 2.78[A]; Preparatory school
graduates (157): 2.61[B]; Recruited athletes (312): 2.79; Prior
enlisted (44): 2.89; Lower 30 percent of admissions class (366):
2.64[A].
Performance score: Average cumulative MPA; Overall (1,216): 2.90;
Females (190): 2.93; Minorities (229): 2.89; Preparatory school
graduates (157): 2.83; Recruited athletes (312): 2.81; Prior enlisted
(44): 2.93; Lower 30 percent of admissions class (366): 2.84.
Performance score: Average order of merit; Overall (1,216): 469;
Females (190): 440; Minorities (229): 545[B]; Preparatory school
graduates (157): 663[B]; Recruited athletes (312): 568[B]; Prior
enlisted (44): 499; Lower 30 percent of admissions class (366): 646[B].
Performance score: Graduation rate; Overall (1,216): 74%; Females
(190): 75%; Minorities (229): 71%; Preparatory school graduates (157):
69%; Recruited athletes (312): 71%; Prior enlisted (44): 66%; Lower 30
percent of admissions class (366): 65%[B].
Source: GAO analysis, from Air Force Academy sources.
[A] Denotes a group average or percentage with a statistically
significant difference greater than 5% from the overall average or
percentage.
[B] Denotes a group average or percentage with a statistically
significant difference greater than 10% from the overall average or
percentage.
[End of table]
Some groups--such as minorities, preparatory school graduates,
recruited athletes, and students in the lower 30 percent of their class
in terms of academic admissions scores--performed at lower levels on
average in all categories than the class as a whole, but these
differences varied between academies and by category and were not
sizable. For example, one of the lowest average academic grade point
averages for the groups we reviewed was 2.61 and the average for the
class as a whole at that academy was 2.93. A 2.0 grade point average is
required to graduate for academic and military averages. Similarly, the
lowest graduation rate for the class we reviewed was 65 percent for the
students in the lower 30 percent of their class in terms of academic
admissions scores at one academy. The average graduation rate for the
class as a whole was 74 percent.
Our analysis of data for the students who entered the academies in 1998
(class of 2002) indicates that admissions scores are generally good
predictors of performance at the academies. Of the admissions scores,
the academic component of the whole person scores was often the best
predictor of academic performance at the academies, and the whole
person scores in their entirety were often the best predictors of
military performance at the academies. Both academic and whole person
admissions scores were good predictors of class rank. In general, whole
person admissions scores were better predictors of graduation rate than
the academic admissions scores alone.
Conclusion:
Although the service academies receive oversight from a number of
organizations and have established guidance for that oversight that
includes the reporting of a wide range of data on academy operations,
without clear and agreed-upon performance goals, there is no objective
yardstick against which to fully measure academy performance and
operations, apart from the officer accessions goals currently used.
Establishment of such performance goals is consistent with the
principles of effective management and would enhance the quality of
oversight already performed by OUSD/P&R, the services, and the academy
boards of visitors, permitting them to more clearly note those areas in
which the academies excel, highlight areas where improvement is
warranted, and achieve the best value for the nation's investment in
the academies.
Recommendation for Executive Action:
To improve DOD oversight of the operations and performance of the
service academies, we recommend that the Secretary of Defense direct
the OUSD/P&R, in concert with the services, to further enhance
performance goals and measures whereby the information required in
annual assessment reports can be better evaluated. These performance
goals should be developed for each academy and, where appropriate, in
common for all academies. The specific goals should coincide with
performance elements agreed upon by the services and OUSD/P&R and might
include such things as graduation rates, demographic composition of
student classes, assessments of officer performance after graduation,
and other performance information already collected by the academies,
including performance characteristics of various groups of students.
Agency Comments:
In comments on a draft of this report, DOD agreed with our
recommendation to further enhance performance goals and measures for
the service academies whereby the information required in annual
assessment reports can be better evaluated. DOD further stated that the
Office of the Under Secretary of Defense for Personnel and Readiness
OUSD/P&R will (1) monitor development of improved goals and measures by
the service academies, to include facilitating the development of
common performance goals where appropriate and (2) update DOD:
Directive 1322.22, Service Academies, as required. DOD's written
comments are included in their entirety in appendix III.
We are sending copies of this report to the appropriate congressional
committees; the Secretaries of Defense, the Army, the Navy, and the Air
Force; and the Director, Office of Management and Budget. We will also
make copies available to others upon request. In addition, the report
will be available at no charge on the GAO Web site at http://
www.gao.gov.
Please contact me on (202) 512-5559 if you or your staff have any
questions concerning this report. Key contributors are listed in
appendix V.
[End of section]
Derek B. Stewart
Director
Defense Capabilities and Management:
Signed by Derek B. Stewart:
[End of section]
Appendix I: Scope and Methodology:
To assess the extent to which DOD oversees the service academies'
operations and performance, we interviewed officials at the Office of
the Under Secretary of Defense for Personnel and Readiness; the Army,
Navy, and Air Force headquarters; and the U.S. Military, U.S. Naval,
and U.S. Air Force Academies. We reviewed documents on service and DOD
oversight criteria and structures, reporting mechanisms, academy
strategic plans, academy annual reports on operations and performance,
boards of visitors' minutes and reports, and superintendents conference
reports. We also attended a U.S. Naval Academy Board of Visitors
meeting at the Naval Academy in December 2002 and a U.S. Military
Academy Board of Visitors meeting in Washington, D.C., in March 2003.
Additionally, we reviewed criteria on the principles of effective
management, such as those found in Standards for Internal Control in
the Federal Government.[Footnote 32]
To assess the extent to which academy applicants are granted waivers
from academic admissions criteria, we interviewed officials from the
Military, Naval, and Air Force Academies and reviewed documents on
admissions policies, standards, and practices. We discussed with
academy officials their execution of the whole person approach,
including how they assess applicants' records, the weights applied to
the various components of the whole person score (academic, leadership,
and physical aptitude), and the justification for points given to
various aspects of an applicant's scores. We also reviewed data from
each academy on trends in academic admissions scores. During site
visits to each academy, we observed the evaluation of applicant
packages for the incoming class of 2007 by academy officials, including
how the whole person approach was applied for admissions scores. We
also observed meetings of senior officials at each academy where
applicants' records were evaluated and final admissions decisions were
made.
To assess the extent to which admissions and academy performance scores
differ between various groups of students, we analyzed admissions
scores and academy performance scores for all students who started at
the three academies in 1998 and should have graduated in 2002. This
represented the most recent group of students for which complete data
were available. We requested and received from each academy a database
that included data on both admission scores and information about
students' performance while attending the academy. We did not
independently assess data reliability, but we obtained assurances about
data completeness, accuracy, and reliability from academy officials
responsible for maintaining data at each academy. We analyzed these
data separately for each academy since each academy calculated
admission scores or performance scores somewhat differently. We
identified six major groups of students common to all academies:
females, minorities, academy preparatory school graduates, recruited
athletes, prior enlisted personnel, and students whose academic
admission scores fell in the lower 30 percent of the entering class (we
chose the latter group in order to capture information on students
whose academic admissions scores may have been lower than might
normally be competitive). Information specifying a student's membership
in each of these groups was provided in the databases from the
academies. To assess differences, we first compared the mean
performance scores for each group to the overall mean for each
performance measure for the entire class. See appendix II for details
on the results of our analysis of the relationships between admissions
and performance scores.
In addition, we assessed the relationship between admissions scores and
performance at the academies by using the whole person admission score
and the academic component of the admissions score. We estimated the
effects of those scores on four measures of performance for students at
the academies: (1) cumulative grade point average (GPA), (2) cumulative
military performance average (MPA), (3) order of merit (class
standing), and (4) graduation rate. We used cumulative GPA upon
graduation as an indicator of academic performance at the academies and
military performance averages upon graduation as an indicator of
military performance at the academies. Order of merit is a measure of
class standing at each academy that combines academic and military
grade performance and is a final rank for each graduating student. At
both the Air Force Academy and the Naval Academy, order of merit is an
actual class rank number. At the Military Academy, however, order of
merit could range between 0 and 4.0 and was given on the same scale as
grade point averages. For each academy, we analyzed the association of
both the academic component scores and whole person admission scores
with each of the performance scores using regression models.
Relationships between the admissions scores and cumulative GPA,
cumulative MPA, and order of merit were estimated using linear
regression models. The relationships between these two admissions
scores and the likelihood of graduating were estimated using logistic
regression models. See appendix II for more details on the results of
those analyses.
Issues related to alleged sexual assaults at the academies fell outside
the scope of our objectives. We conducted our work from October 2002
through May 2003 in accordance with generally accepted government
auditing standards.
[End of section]
Appendix II: Results of Statistical Analysis of Class of 2002
Admissions and Academy Performance Scores:
This appendix provides the results of our analyses of both admissions
and performance scores for the class of 2002 at the U.S. Military
Academy, the U.S. Naval Academy, and the U.S. Air Force Academy.
Admissions and Performance Scores:
We obtained data from all three service academies that included
information on admissions scores (academic and whole person),
performance scores while at the academy (cumulative academic grade
point average, military performance average, and order of merit),
attrition information where applicable, and various demographic
characteristics for all students entering each academy in 1998. Table
10 shows the minimum, maximum and average admissions and performance
scores for students at each academy. Table 11 shows graduation rates at
each academy.
Table 10: Admissions and Academy Performance Scores for the Class of
2002:
Academic Admissions score; Military Academy: Average: 600; Military
Academy: Min: 430; Military Academy: Max: 791; Naval Academy:
Average: 618; Naval Academy: Min: 440; Naval Academy: Max: 788;
Air Force Academy: Average: 3,202; Air Force Academy: Min:
2,492; Air Force Academy: Max: 4,005.
Whole person score; Military Academy: Average: 6,006; Military Academy:
Min: 4,587; Military Academy: Max: 7,188; Naval Academy:
Average: 65,732; Naval Academy: Min: 51,651; Naval Academy: Max:
82,250; Air Force Academy: Average: 798; Air Force Academy:
Min: 655; Air Force Academy: Max: 931.
Cum. GPA; Military Academy: Average: 2.99; Military Academy: Min: 1.97;
Military Academy: Max: 4.19; Naval Academy: Average: 2.97;
Naval Academy: Min: 2.03; Naval Academy: Max: 4.00; Air Force
Academy: Average: 2.93; Air Force Academy: Min: 2.06; Air Force
Academy: Max: 3.97.
Cum. MPA; Military Academy: Average: 3.28; Military Academy: Min: 2.09;
Military Academy: Max: 3.99; Naval Academy: Average: 3.12;
Naval Academy: Min: 2.17; Naval Academy: Max: 3.85; Air Force
Academy: Average: 2.90; Air Force Academy: Min: 2.32; Air Force
Academy: Max: 3.92.
Order of merit; Military Academy: Average: 3.03; Military Academy: Min:
1.30; Military Academy: Max: 3.92; Naval Academy: Average:
489; Naval Academy: Min: 1; Naval Academy: Max: 977; Air Force
Academy: Average: 469; Air Force Academy: Min: 1; Air Force Academy:
Max: 929.
Source: GAO analysis, from DOD sources.
Note: For the U.S. Air Force Academy, an additional step during the
selection panel process results in a lower whole person score than the
component parts.
[End of table]
Table 11: Number of Students Graduating and Graduation Rates for the
Class of 2002:
Graduation rate; Military Academy: Number: 968; Military Academy:
Percent: 78%; Naval Academy: Number: 977; Naval Academy:
Percent: 80%; Air Force Academy: Number: 894; Air Force
Academy: Percent: 74%.
Source: GAO analysis, from DOD sources.
[End of table]
Next, we compared the average admissions scores, performance scores,
and graduation rates of the six student groups to these overall scores
and rates. Tables 12, 13, and 14 show the average admission scores and
the four measures of student performance for the overall sample, and
for the six student groups, for each of the academies. Because we have
data for the population of students in this class and there is no
sampling error, the standard error of these estimates are small and
differences that could be considered small in magnitude may in fact be
statistically significant. In the tables below, differences that are
statistically significant (p
