Defense Transportation

The Army began a pilot project in 1997 to test an alternative approach to providing relocation services to its military personnel stationed at Hunter Army Airfield in Savannah, Georgia. The Army undertook this effort to address long-standing problems with the current personal property program. The Defense Department (DOD) has two other pilot programs underway to test different approaches to improving its personal property program and is proposing a fourth pilot. To determine which pilot, or portion thereof, could provide better long-term results, the U.S. Transportation Command is responsible for overseeing all personal property pilot tests and recommending the follow-on course of action. GAO is required to validate the results and savings achieved before DOD expands any of its personal property pilot projects. This report supplements a March 1999 testimony (GAO/T-NSIAD-99-106) and addresses the results of GAO's 12-month Hunter pilot test and lessons for evaluating other pilots. GAO (1) assesses the Army's evaluation methodology of the Hunter Pilot, including the validity of data and reported results, and (2) determines the status of all ongoing and planned pilot projects and the adequacy of DOD's plans to evaluate them.

GAO noted that: (1) although the Army reported that the Hunter pilot was a success, GAO found that most of the results of its evaluation were inconclusive; (2) while the Hunter pilot provides services and benefits that were not previously available during the moving process, GAO was unable to validate all reported results of the Army's evaluation of this pilot because of weaknesses in the evaluation methodology and data; (3) specifically, because of the methods used in conducting the customer surveys, GAO could not confirm that customer satisfaction improved; (4) also, due to weaknesses in the Army's methodology and data reliability, GAO could not validate the extent to which pilot costs exceeded baseline costs; (5) however, GAO was able to confirm that 33 percent of the pilot shipments were awarded to small business carriers and agents; (6) further, lessons learned by the Army in developing an evaluation plan, conducting the pilot test, and evaluating results can provide useful information to DOD as it conducts and assesses other pilot efforts; (7) the Navy and the Military Traffic Management Command each have a personal property pilot project underway, and DOD is proposing a fourth pilot to test different approaches to improve its personal property program; (8) as a result, DOD will be running multiple pilots concurrently, with different goals, objectives, and expected outcomes; (9) U.S. Transportation Command (USTRANSCOM) is tasked with evaluating the results of the pilots and using that information to recommend a redesigned Department-wide relocation program; (10) however, DOD has not yet determined how many approaches will ultimately be tested and the milestones for completing the pilots' evaluation and implementing an improved process, nor has it assured itself that a methodologically sound evaluation process is in place to execute this process; (11) improving DOD's personal property program has been a slow, complex process; (12) DOD and the services have spent a large amount of time and effort to dramatically change the quality of services military personnel receive; and (13) GAO supports pilots as a tool to test different approaches.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone: