Federal Bureau of Investigation

Weak Controls over Trilogy Project Led to Payment of Questionable Contractor Costs and Missing Assets Gao ID: GAO-06-306 February 28, 2006

The Trilogy project--initiated in 2001--is the Federal Bureau of Investigation's (FBI) largest information technology (IT) upgrade to date. While ultimately successful in providing updated IT infrastructure and systems, Trilogy was not a success with regard to upgrading FBI's investigative applications. Further, the project was plagued with missed milestones and escalating costs, which eventually totaled nearly $537 million. In light of these events, Congress asked GAO to determine whether (1) internal controls provided reasonable assurance that improper payment of unallowable contractor costs would not be made or would be detected in the normal course of business, (2) payments to contractors were properly supported as a valid use of government funds, and (3) FBI maintained proper accountability for assets purchased with Trilogy project funds.

FBI's review and approval process for Trilogy contractor invoices, which included a review role for the General Services Administration (GSA) as contracting agency, did not provide an adequate basis to verify that goods and services billed were actually received and that the amounts billed were appropriate, leaving FBI highly vulnerable to payments of unallowable costs. This vulnerability is demonstrated by FBI's payment of about $10.1 million in questionable contractor costs we identified using data mining, document analysis, and other forensic auditing techniques. These costs included first-class travel and other excessive airfare costs, incorrect charges for overtime hours, potentially overcharged labor rates, and charges for which the contractors could not provide adequate supporting documentation to substantiate the costs purportedly incurred. FBI also failed to establish controls to maintain accountability over equipment purchased for the Trilogy project. These control lapses resulted in more than 1,200 missing pieces of equipment valued at approximately $7.6 million that GAO identified as part of its review. In addition, in its own inventory counts, FBI identified 37 pieces of Trilogy equipment valued at approximately $167,000 that had been lost or stolen.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-06-306, Federal Bureau of Investigation: Weak Controls over Trilogy Project Led to Payment of Questionable Contractor Costs and Missing Assets This is the accessible text file for GAO report number GAO-06-306 entitled 'Federal Bureau of Investigation: Weak Controls over Trilogy Project Led to Payment of Questionable Contractor Costs and Missing Assets' which was released on March 20, 2006. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Requesters: February 2006: Federal Bureau of Investigation: Weak Controls over Trilogy Project Led to Payment of Questionable Contractor Costs and Missing Assets: GAO-06-306: GAO Highlights: Highlights of GAO-06-306, a report to congressional requesters: Why GAO Did This Study: The Trilogy project”initiated in 2001”is the Federal Bureau of Investigation‘s (FBI) largest information technology (IT) upgrade to date. While ultimately successful in providing updated IT infrastructure and systems, Trilogy was not a success with regard to upgrading FBI‘s investigative applications. Further, the project was plagued with missed milestones and escalating costs, which eventually totaled nearly $537 million. In light of these events, Congress asked GAO to determine whether (1) internal controls provided reasonable assurance that improper payment of unallowable contractor costs would not be made or would be detected in the normal course of business, (2) payments to contractors were properly supported as a valid use of government funds, and (3) FBI maintained proper accountability for assets purchased with Trilogy project funds. What GAO Found: FBI‘s review and approval process for Trilogy contractor invoices, which included a review role for the General Services Administration (GSA) as contracting agency, did not provide an adequate basis to verify that goods and services billed were actually received and that the amounts billed were appropriate, leaving FBI highly vulnerable to payments of unallowable costs. This vulnerability is demonstrated by FBI‘s payment of about $10.1 million in questionable contractor costs we identified using data mining, document analysis, and other forensic auditing techniques. These costs included first-class travel and other excessive airfare costs, incorrect charges for overtime hours, potentially overcharged labor rates, and charges for which the contractors could not provide adequate supporting documentation to substantiate the costs purportedly incurred. FBI also failed to establish controls to maintain accountability over equipment purchased for the Trilogy project. These control lapses resulted in more than 1,200 missing pieces of equipment valued at approximately $7.6 million that GAO identified as part of its review. In addition, in its own inventory counts, FBI identified 37 pieces of Trilogy equipment valued at approximately $167,000 that had been lost or stolen. The table below summarizes questionable contractor costs and missing assets that GAO identified. Questionable Costs and Missing Assets: Issues identified: First-class travel; Amount (in thousands): $20.0. Issues identified: Excessive air travel costs; Amount (in thousands): $49.8. Issues identified: Excess overtime charges; Amount (in thousands): $400.0. Issues identified: Potential overcharging of labor rates; Amount (in thousands): $2,100.0. Issues identified: Inadequately supported subcontractor labor costs; Amount (in thousands): $1,957.9. Issues identified: Inadequately supported other direct costs; Amount (in thousands): $5,508.3. Issues identified: Duplicate payment of subcontractor labor invoice; Amount (in thousands): $26.3. Issues identified: Total questionable costs; Amount (in thousands): $10,062.3. Issues identified: 1,205 pieces of missing equipment; Amount (in thousands): $7,607.1. Source: GAO. [End of table] Given the poor control environment and the fact that GAO reviewed only selected FBI payments to Trilogy contractors, other questionable contractor costs may have been paid that have not been identified. If these control weaknesses go uncorrected, future contracts, including those related to Sentinel”FBI‘s new electronic information management system initiative”will be highly exposed to improper payments. In addition, the lack of accountability for Trilogy equipment calls into question FBI‘s ability to adequately safeguard its existing assets as well as those it may acquire in the future. What GAO Recommends: GAO makes 27 recommendations to help improve (1) FBI‘s and GSA‘s controls over their invoice review and approval processes and to address questionable billing issues, and (2) FBI‘s accountability for assets. FBI concurred with our recommendations. While GSA accepted our recommendations, it did not believe that 1 of them was needed and expressed concern with some of our findings. GAO reaffirms its position on all of its findings and recommendations. www.gao.gov/cgi-bin/getrpt?GAO-06-306. To view the full product, including the scope and methodology, click on the link above. For more information, contact Linda Calbom at (202) 512- 9508 or calboml@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: Insufficient Invoice Review and Approval Process Increased FBI's Vulnerability to Payment of Unallowable Contractor Costs: Some Payments Made to Contractors Were for Questionable Costs: Major Lapses in Accountability Resulted in Millions of Dollars of Missing Trilogy Equipment: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendixes: Appendix I: Key Trilogy Milestones: Appendix II: Scope and Methodology: Validity of Payments: FBI's Asset Accountability: Appendix III: Comments from the Federal Bureau of Investigation: Appendix IV: Comments from the General Services Administration: Appendix V: GAO Contacts and Staff Acknowledgments: Tables: Table 1: Payments for Trilogy by Contractor and Category (in millions): Table 2: Examples of CSC's Potentially Unallowable First-Class Travel: Table 3: Examples of Questionable Excessive Airfare Travel Costs: Table 4: Other Questionable Costs Paid by FBI: Table 5: Analysis of Time Taken by FBI to Enter CSC-Purchased Assets into PMA: Table 6: Trilogy-Purchased Items Not Located by FBI: Figures: Figure 1: Invoice Review and Approval Process: Figure 2: CSC E-mail Approval of Subcontractor ODC Charge: Figure 3: Example of CSC ODC Invoice: Figure 4: Key Trilogy Milestones: Letter February 28, 2006: The Honorable Arlen Specter: Chairman: The Honorable Patrick J. Leahy: Ranking Minority Member: Committee on the Judiciary: United States Senate: The Honorable Richard J. Durbin: United States Senate: The Honorable Charles E. Grassley: United States Senate: The Honorable Orrin G. Hatch: United States Senate: For several years Congress recognized that the Federal Bureau of Investigation's (FBI) information technology (IT) systems were archaic and inadequate for efficiently and effectively investigating criminal cases. FBI recognized the need to modernize its IT systems before the September 11, 2001, terrorist attacks, but those events underscored FBI's need to improve its ability to effectively retrieve, analyze, and share investigative information necessary to carry out its mission. Initiated in mid-2001, Trilogy--FBI's largest IT upgrade to date--was intended to modernize FBI's IT infrastructure and systems and provide needed applications to help FBI agents, analysts, and others do their jobs. The Trilogy project consisted of two primary efforts: an IT infrastructure update and an upgrade of FBI's investigative applications. While ultimately successful in providing the infrastructure update, the project was not a success with regard to upgrading the investigative applications. Further, the project experienced numerous schedule delays and cost increases.[Footnote 1] Project costs, which were originally estimated at approximately $380 million, eventually escalated to approximately $537 million. Although the scheduled completion date for the overall Trilogy project was June 2004, after September 11, 2001, FBI required an accelerated deployment plan and moved up the expected completion dates. The completion date for the portion of Trilogy related to upgrading FBI's IT infrastructure was accelerated from May 2004 to July 2002. After several delays, the upgrade was completed in April 2004, a month before the original due date. While the overall scheduled completion date for the investigative application upgrades, which became known as the Virtual Case File (VCF), was originally June 2004, the due date for the first VCF deliverable was accelerated to December 2003. However, in July 2004 the VCF portion of the Trilogy project was scaled back after the completion of the project was determined to be infeasible and cost prohibitive as originally envisioned. The Department of Justice (DOJ) Office of Inspector General (OIG) has reported numerous issues that contributed to the cost increases and delays, including poorly defined and slowly evolving design requirements, contracting weaknesses, unrealistic task scheduling, and lack of management continuity and oversight for tracking and overseeing costs effectively.[Footnote 2] GAO also reported on weaknesses in FBI's IT systems development and management capabilities, including contractor oversight.[Footnote 3] Because of these issues, you asked us to audit the costs of the Trilogy project, the majority of which represented the purchase of goods and services from contractors. Our objectives were to determine whether (1) FBI's internal controls provided reasonable assurance that payment of unallowable contractor costs would not be made or would be detected in the normal course of business,[Footnote 4] (2) FBI's payments to contractors were properly supported as a valid use of government funds, and (3) FBI maintained proper accountability for assets purchased with Trilogy project funds. To address these objectives, we used various internal control standards and guidance[Footnote 5] as a basis to assess FBI's internal controls over the payments made with Trilogy funds. We also reviewed FBI policy and procedure manuals; applicable federal regulations, including the Federal Acquisition Regulation (FAR),[Footnote 6] Federal Travel Regulation,[Footnote 7] and Joint Travel Regulations (JTR);[Footnote 8] prior GAO and DOJ OIG reports on Trilogy issues; Trilogy contract documents and interagency agreements; contractor invoices; and other documentation supporting goods provided and services rendered. We performed data mining and forensic auditing techniques to select transactions to determine whether payments to contractors were properly supported as a valid use of government funds. We tested accountable property to determine whether assets were entered in FBI's property system and conducted a physical observation of selected assets to validate their existence. In addition, we conducted interviews with officials from FBI, General Services Administration's (GSA) Federal Systems Integration and Management Center (FEDSIM), Department of the Interior (DOI), and Trilogy contractors. We also performed walkthroughs to gain an understanding of the processes used to review and approve invoices and account for property. While we identified some payments for questionable contractor costs,[Footnote 9] our work was not designed to identify all questionable payments or to estimate their extent. We provided FBI a draft of this report and GSA a draft of applicable sections of this report for review and comment. FBI and GSA provided written comments, which are reprinted in appendixes III and IV, respectively. FBI and GSA also provided technical comments, which we have incorporated as appropriate. We also discussed with Trilogy contractors any findings that related to them. We performed our work in accordance with generally accepted government auditing standards in Washington D.C. and at two FBI field sites and various other GSA and contractor locations in Virginia from May 2004 through December 2005. Our scope and methodology are discussed in greater detail in appendix II. Results in Brief: FBI's internal controls did not provide reasonable assurance that payments to contractors for unallowable costs would not be made or would be detected in the normal course of business. Our review found that FBI's review and approval process for Trilogy contractor invoices, which included GSA's review in its role as contracting agency, did not provide an adequate basis to verify that goods and services billed were actually received by FBI or that the amounts billed were appropriate. This occurred in part because responsibility for the review and approval of invoices was not clearly defined in the contracts and interagency agreements related to Trilogy project oversight. In addition, certain contactor invoices lacked certain detailed information required by the Trilogy task orders and other additional information that would be needed to facilitate an adequate invoice review process. Despite this, invoices were paid without requests for additional supporting documentation necessary to validate the charges. These weaknesses made FBI highly vulnerable to payments of unallowable and questionable costs with Trilogy funds. Until significant improvements are made, these vulnerabilities will continue for future projects where FBI uses contractors for the delivery and deployment of goods and services. We used forensic auditing techniques, including data mining and document analysis, to assess the validity of selected payments and identified $10.1 million of questionable contractor costs paid by FBI. We found instances of first-class travel and other excessive airfare costs, incorrect billings for overtime hours worked, potential overcharging of labor rates, and other questionable costs. For example, one contractor could not provide adequate documentation to substantiate about $5.5 million of subcontractor charges for other direct costs billed to FBI. Given FBI's poor control environment over invoice payments and the fact that we reviewed only selected FBI payments to Trilogy contractors, other questionable costs may have been paid for that have not been identified. Further, if these weaknesses go uncorrected, future contracts, including those related to FBI's new electronic information management system initiative, referred to as Sentinel, will be highly exposed to improper payments. FBI did not maintain adequate accountability for all computer equipment purchased for the Trilogy project. FBI relied extensively on contractors to account for Trilogy assets while they were being purchased, warehoused, and installed. However, FBI did not have controls or data to verify the accuracy and completeness of the contractor records it ultimately relied on and to ensure that it received all the items purchased through its contractors. Moreover, once FBI took possession of the Trilogy equipment, it did not establish adequate physical control over the assets. FBI failed to record accountable assets--equipment with a value of $1,000 or more, or deemed by FBI to be susceptible to theft--into its property system in a timely manner, did not properly use its bar codes to individually track accountable assets, and did not effectively use its inventory process to identify all potentially missing assets. These breakdowns in control over Trilogy assets created an environment in which equipment could be lost or stolen without detection. Given the serious nature of these control weaknesses, we performed additional test work to determine whether all accountable assets purchased with Trilogy funds could be accounted for by FBI. FBI was unable to locate over 1,200 of these assets, which we estimate are valued at approximately $7.6 million, including items such as computer desktops, laptops, printers, and servers. In addition to the items we found missing, as a result of its physical inventory procedures, FBI reported 37 pieces of contractor-purchased Trilogy equipment valued at about $167,000 that had been lost or stolen. Due to the significant weaknesses we identified in FBI's Trilogy property controls, the actual amount of missing or stolen equipment could be even higher. Until FBI strengthens its asset accountability controls it will remain highly vulnerable to continued loss of existing assets, as well as those it may acquire in the future. We are making 27 recommendations to address the issues identified in this report. Regarding FBI's and GSA's processes for reviewing and approving contractor invoices, we are making 6 recommendations to FBI and 5 to GSA to develop or strengthen these types of internal control procedures. We are making 4 additional recommendations to GSA in coordination with FBI, to take actions to resolve certain of the questionable costs we identified. And we are making 12 other recommendations to help FBI improve its accountability over existing Trilogy assets and those that will be purchased in connection with future projects such as Sentinel. In written comments on a draft of this report, FBI stated that it concurred with our recommendations and that it has made and continues to make significant structural and procedural changes to address our recommendations. FBI also provided additional information related to Trilogy assets we identified as missing. In written comments on a draft of applicable sections of this report, while GSA stated that it accepted our recommendations, it did not believe that 1 of them was needed, and described some of the improvements to its internal controls and other business process changes already implemented. GSA also expressed concern with some of our observations and conclusions related to the invoice review and approval process and our analysis of airfare costs. We continue to believe that our report is accurate and that all of the recommendations should be implemented. Our responses to these comments are provided in the Agency Comments and Our Evaluation section of this report and in appendix IV, immediately following the reprinted GSA comments. Background: Recognizing the need to modernize its IT systems, FBI proposed a major technology upgrade plan to Congress in September 2000. FBI's Information Technology Upgrade Project, which FBI subsequently renamed Trilogy, was FBI's largest automated information systems initiative to date. Trilogy consisted of three parts: (1) the Information Presentation Component (IPC) to upgrade FBI's computer hardware and software, (2) the Transportation Network Component (TNC) to upgrade FBI's communication network, and (3) the User Application Component (UAC) to upgrade and consolidate FBI's five most important investigative applications. To expedite the contracting process, FBI entered into an interagency agreement with GSA to support FBI's use of the FEDSIM Millennia governmentwide acquisition contract[Footnote 10] for the implementation of Trilogy's three functional components, IPC, TNC, and UAC. FEDSIM, serving as contracting agency, was to provide all contract administrative services necessary to support the task orders. Because the Trilogy project was so large, DOJ required FBI to use two contractors for the three Trilogy components. FBI combined the IPC and TNC portions of Trilogy into one task order because both components involved physical infrastructure enhancements. IPC provided for new desktop computers, servers, and commercial-off-the-shelf automation software, including Web-browser and e-mail to enhance usability by the agents. TNC upgraded the complete communication infrastructure, including high-capacity wide-area and local-area networks, authorization security, and encryption of data transmission and storage. The IPC/TNC task order was awarded in May 2001 to DynCorp (now Computer Sciences Corporation (CSC)).[Footnote 11] The IPC/TNC upgrades would provide the physical infrastructure needed to run the applications developed under UAC, the third Trilogy component. The third component of Trilogy--the UAC task order--was awarded in June 2001 to Science Applications International Corporation (SAIC). The goal of UAC was to replace FBI's paper case files with electronic files and improve efficiency. The heart of the UAC portion became the development of the VCF system to replace the obsolete Automated Case Support system, FBI's primary investigative application that uploads and stores case files electronically. The above two Trilogy contracts[Footnote 12] were awarded on a cost- plus-award fee basis for labor charges, meaning that the contractor's costs incurred are reimbursed and fees[Footnote 13] may be awarded to the contractor based on performance. The FAR states that cost- reimbursement type contracts may only be used if appropriate government surveillance during performance will provide reasonable assurance that efficient methods and effective cost controls are used. The aspects of these contracts related to the purchase of equipment were based on fixed-price arrangements, meaning that a set price for the equipment is agreed to up front. In addition to the two primary contracts discussed above, FBI awarded two additional contracts to assist with the technical oversight, monitoring, and integration of the two primary Trilogy contracts described above. The first of the two additional contracts was awarded in February 2001, also through GSA FEDSIM, to Mitretek for Systems Engineering and Technical Assistance (SETA) services.[Footnote 14] Under the SETA contract, Mitretek was required to assist FBI with a wide array of tasks, including program and contract management, fiscal and budgetary oversight, cost estimating, and several other technical aspects of the Trilogy project. The second of the two additional contracts was awarded to SAIC for the integration[Footnote 15] of the three Trilogy components. [Footnote 16] In July 2004, the VCF was scaled back to the Initial Operating Capability and the remaining deliverables were cancelled after the (1) initial deliverable was rejected by FBI and (2) VCF was determined to be infeasible and cost prohibitive to implement as originally envisioned. After a 90-day limited pilot that ended in March 2005, VCF offline and the pilot results were then to be analyzed by FBI for requirements development of its new electronic information management system initiative. In August 2005, FBI released a solicitation for proposals to develop FBI's new electronic information management system, referred to as Sentinel. The solicitation was sent to more than 40 eligible companies under a National Institutes of Health governmentwide acquisition contract. Similar to VCF, the goal of Sentinel is to replace FBI's legacy case management capabilities with an integrated, paperless file management and workflow system. FBI's Asset Accountability Procedures: According to FBI policy, assets valued at $1,000 or more, as well as certain sensitive items, such as firearms, laptop computers, and central processing units, are considered to be "accountable" assets, regardless of cost, and must be accounted for individually in FBI's Property Management Application (PMA). PMA is an automated management system that allows FBI to track the cost, location, and history of its accountable assets. PMA includes a variety of data fields to identify each item, including the acquisition date, received date, acquisition cost, last inventory date, bar code number, serial number, cost center for the office where the item is located, description of the item, and other information. Ongoing deficiencies in FBI's management of property have been identified by DOJ's OIG and FBI's independent financial statement auditor. In August 2002, the DOJ OIG issued a report that revealed significant problems with FBI's management of laptop computers, including findings that FBI did not reconcile its property management data with purchase data from its accounting system, did not have an inventory record for accountable assets not in PMA that were lost or stolen, and could not verify whether the number of items purchased agreed with the number of items recorded in PMA.[Footnote 17] Additionally, in September 2004, the DOJ OIG reported on weaknesses in FBI's controls over nonaccountable property at FBI's Baltimore field office after an employee pleaded guilty to the theft and sale of FBI photography equipment.[Footnote 18] Annually, since fiscal year 1999, FBI's independent financial statement auditors have identified internal control weaknesses in the area of property management. They specifically reported that FBI needed to improve its procedures related to the timely and accurate recording, reconciling, and reporting of property and equipment in PMA. Internal Control: Internal control is a major part of managing any organization. As required by 31 U.S.C. 3512(c),(d), commonly referred to as the Federal Managers' Financial Integrity Act of 1982, the Comptroller General issues standards for internal control in the federal government.[Footnote 19] These standards provide the overall framework for establishing and maintaining internal control and for identifying and addressing major performance and management challenges and areas at greatest risk of fraud, waste, abuse, and mismanagement. According to these standards, internal control comprises the plans, methods, and procedures used to meet missions, goals, and objectives. Internal control is the first line of defense in safeguarding assets and preventing and detecting fraud and errors. Internal control, which is synonymous with management control, helps government program managers achieve desired results through effective stewardship of public resources. Control activities are the policies, procedures, techniques, and mechanisms that enforce management's directives and help ensure that actions are taken to address risks. Control activities are an integral part of an entity's planning, implementing, reviewing, and accountability for stewardship of government resources and achieving effective results. They include a wide range of diverse activities. Some examples of control activities include (1) establishing physical controls over vulnerable assets to reduce the risk of loss or unauthorized use and periodically counting and comparing such assets to control records; (2) ensuring that documentation and records are properly managed and maintained and that transactions are appropriately documented and readily available for examination; (3) assigning accountability for the custody and use of resources and records to help reduce the risk of errors, fraud, misuse, or unauthorized alteration; and (4) implementing management level reviews at the functional level to ensure that appropriate control activities are being employed, such as reconciliations of summary information to supporting detail. Insufficient Invoice Review and Approval Process Increased FBI's Vulnerability to Payment of Unallowable Contractor Costs: FBI's review and approval process for Trilogy contractor invoices, which was carried out by a review team consisting of officials from FBI, GSA, and Mitretek, did not provide an adequate basis to verify that goods and services billed were actually received by FBI or that payments were for allowable costs. This occurred in part because responsibility for the review and approval of invoices was not clearly defined in the Mitretek contract and in the interagency agreements related to Trilogy project oversight. In addition, contactor invoices frequently lacked detailed information required by the contracts and other additional information that would be needed to facilitate an adequate invoice review process. Despite this, invoices were paid without requests for additional supporting documentation necessary to determine the validity of the charges. These weaknesses in FBI's review and approval process made the agency highly vulnerable to payment of unallowable or questionable contractor costs. Invoices Were Approved for Payment without Validation that Goods and Services Were Received: While the review and approval process differed for each contractor and type of invoice charge, in general the process carried out by the review team lacked key procedures to reasonably ensure that goods and services billed were actually received by FBI or that the amounts billed and paid were for allowable costs. Internal control guidance requires agencies to establish controls that reasonably ensure, among other things, that funds, property, and other assets are safeguarded against waste, loss, unauthorized use, or misappropriation.[Footnote 20] Contractor invoices included costs for labor, including related overhead costs; travel; other direct costs (ODC); subcontractor labor; and purchased equipment. Table 1 provides a summary of total payments made to Trilogy contractors for these categories, as well as total Trilogy costs in each category. Table 1: Payments for Trilogy by Contractor and Category (in millions): Category: Labor; DynCorp/CSC: $2.9; SAIC: $67.7; Mitretek: $19.5; Contractor total: $90.1; Trilogy total[D]: $102.3; Contractor percentage of Trilogy total: 88%. Category: Subcontractor labor; DynCorp/CSC: $116.2; SAIC: $46.9; Contractor total: $163.1; Trilogy total[D]: $163.1; Contractor percentage of Trilogy total: 100%. Category: Travel[A]; DynCorp/CSC: $9.5; SAIC: $0.3; Mitretek: $0.1; Contractor total: $9.9; Trilogy total[D]: $13.4; Contractor percentage of Trilogy total: 74%. Category: Other direct costs[B]; DynCorp/CSC: $8.9; SAIC: $0.5; Mitretek: $1.9; Contractor total: $11.3; Trilogy total[D]: $11.3; Contractor percentage of Trilogy total: 100%. Category: Equipment; DynCorp/CSC: $115.7; SAIC: $1.7; Contractor total: $117.4; Trilogy total[D]: $221.3; Contractor percentage of Trilogy total: 53%. Category: Other[C]; DynCorp/CSC: $18.5; SAIC: $5.0; Mitretek: $1.1; Contractor total: $24.6; Trilogy total[D]: $25.5; Contractor percentage of Trilogy total: 96%. Category: Totals; DynCorp/CSC: $271.7; SAIC: $122.1; Mitretek: $22.6; Contractor total: $416.4; Trilogy total[D]: $536.9; Contractor percentage of Trilogy total: 78%. Source: GAO analysis of contractor invoices, Mitretek Spend Plan, and FBI records. [A] Subcontractor charges for travel were included in CSC and SAIC travel invoices and are included under the travel category. [B] Subcontractor charges for ODC were included in CSC's ODC invoices and are included under the ODC category. [C] For CSC, the "other" category represents miscellaneous maintenance charges and $10 million in awards and fees. For SAIC, the "other" category represents $5 million in awards and fees. For Mitretek, the "other" category represents $1.1 million in fees. We did not assess the propriety of these payments. [D] Trilogy costs beyond those billed by contractors primarily related to direct purchases of equipment by FBI. [End of table] Each member of the review team--which included personnel from FBI; GSA, the contracting agency; and Mitretek--was to perform some level of review of the invoices submitted by the contractors for payment. During the project, each of the review team members, at times, worked on-site with the contractors. As is discussed later, the specific roles of each party were not clearly defined, which limited the effectiveness of the invoice review and approval process. Figure 1 illustrates this invoice review and approval process. Figure 1: Invoice Review and Approval Process: [See PDF for image] [A] GSA facilitated the payment of contractor invoices and was subsequently reimbursed by FBI with funds appropriated for the Trilogy project. [End of figure] Our review disclosed serious gaps in the review process for each of the major categories of contractor costs, as follows. Labor--According to GSA, it typically reviewed labor charges by looking for unusual or excessive hours worked or rates charged and recalculating some amounts to ensure mathematical accuracy. GSA also stated that its personnel generally compared average fully burdened labor rates (labor, overhead, fringe benefits, and general and administration costs) charged to ceiling rates (maximums) established in the Trilogy contracts. However, GSA was not able to provide us with an explanation for or evidence of how they resolved clearly questionable labor charges we identified, including hours billed far in excess of a normal pay period. For example, we identified one individual who charged 371 hours for one 4-week period (an average of 93 hours per week) and 359 in the following 5-week period (an average of 70 hours per week). There was no evidence that GSA had questioned whether these seemingly excessive hours were valid. GSA stated that these types of issues were usually resolved on the telephone and therefore they usually did not maintain any documentation of their inquiries. On-site members of the review team indicated that they generally knew the contractor employees working on the project and reviewed the hours billed for reasonableness. However, the review team did not have a systematic process in place to help ensure that individuals listed on invoices had actually worked on Trilogy the number of hours being billed or that the job classifications and related billing rates were appropriate. In addition, there was no documented assessment of whether the overall hours being billed for a particular activity were in line with expectations. Subcontractor Labor--The review team paid contractor invoices for subcontractor labor without any attempt to assess the validity of the charges. The GSA official responsible for paying the invoices stated that the review team relied on the contractors to properly bill for the costs related to their subcontractors and to validate the subcontractor invoices. However, the review team had no process in place to assess whether or not the contractors were properly validating their subcontractor labor charges or to assess the allowability of those charges. In addition, we found that CSC, which billed the bulk (i.e., about $116 million) of the subcontractor labor costs,[Footnote 21] did not always have sufficient documentation of subcontractor charges to enable CSC, or anyone else, to perform any assessment of the allowability of those costs. For example, the only supporting documentation CSC could provide us for about $2 million in subcontractor labor charges we selected for review were subcontractor invoices that lacked some of the basic information needed to assess the labor costs, such as the names of the subcontractor employees, hours billed, or individual labor rates. Travel--These charges were reviewed differently by the review team for SAIC and CSC invoices. For SAIC travel, GSA told us they compared invoiced amounts to travel authorizations and verified the per diem and lodging rates in the authorizations to those prescribed under the Federal Travel Regulation. However, travel authorizations were not always submitted and approved before travel occurred and in some cases were based on actual amounts. The review team told us that they reviewed SAIC travel vouchers or receipts in a few instances over 4 years when amounts billed were higher than expected to verify the amounts charged on the travel invoices. However, there was no systematic process to review travel costs billed to the Trilogy project. For CSC travel, because CSC's travel authorizations did not include details by employee or the estimated cost for each trip and frequently covered several trips, the GSA official who paid the invoices told us she relied on members of the review team that worked on-site to review the travel invoices. These on-site review team officials indicated that their review process was based on their general understanding of who was traveling. However, we determined that no one on the review team obtained travel vouchers or receipts to verify that amounts billed by CSC were a necessary and proper charge to the Trilogy project and were reasonable based on the location and length of travel required. Other Direct Costs (ODC)--These charges were paid without validation of the actual amounts included in the invoices. The review team relied on contractors to obtain purchase orders for ODC charges. For SAIC ODC invoices, the review team generally tracked actual charges billed on invoices compared to purchase order amounts. However, there was no review of receipts or other documentation to validate the actual charges on invoices. CSC ODC invoices were paid without matching the charges to a purchase order or documentation of the actual cost incurred. Therefore, the review team had no basis for confidence that CSC ODC charges were approved ahead of time or appropriately billed. CSC ODC charges also included subcontractor ODC. We asked CSC for supporting documentation for selected subcontractor ODC and found that CSC's only support was subcontractor invoices that included only a brief description of the nature of the charge and the amount. No supporting receipts or other documentation necessary to verify the charges was provided. For example, CSC billed FBI for ODC of $456,211 on an invoice submitted in November 2003. The only description on the invoice for these charges was "other direct costs." We requested from CSC any documentation they had in their files to support this charge from its subcontractor, CACI Inc. - Federal (CACI). CSC was able to provide an invoice with one line entitled "facilities/materials" and a spreadsheet with a general summary of the charges. Further, the e-mail exchange presented in figure 2 shows that CSC recognized that they did not have enough detail to review the ODC charge, but approved the invoice anyway. As noted below, the final entry in the exchange is, "It's not what we asked for but at this point it doesn't really matter. Approve it." Figure 2: CSC E-mail Approval of Subcontractor ODC Charge: [See PDF for image] [End of figure] Equipment--Charges for equipment purchased by contractors and billed to FBI were reviewed merely by tracking the total cost of equipment invoices to ensure that the total amount did not exceed the approved amount on purchase orders. However, neither GSA, FBI, nor Mitretek performed procedures to ensure that individual equipment items billed by the contractors were actually received before payment. Discussions with the contractors revealed that this was a high-risk area because some of the invoices they submitted were for equipment that had not yet been delivered to FBI. The review team approved and paid these invoices without question. In addition, FBI purchased some IPC/TNC equipment directly from vendors and delivered the equipment to contractor locations, but did not have a mechanism in place to physically verify receipt of that equipment at FBI sites before paying the related invoices. There was also no subsequent verification by the review team that all equipment purchased through contractors and vendors was ultimately received by FBI. Invoice Review Responsibilities Were Not Clearly Defined: The insufficient invoice review and approval process was at least in part the result of a lack of clarity in the interagency agreement between FBI and GSA FEDSIM, as well as in FBI's oversight contract with Mitretek. We have identified the management of interagency contracting as a high-risk area, in part because it is not always clear with whom the responsibility lies for critical management functions in the interagency contracting process, including contract oversight.[Footnote 22] The lack of clarity in roles and responsibilities was evident in our interviews with the review team, where each party indicated that they believed another party was responsible for a more detailed review. While contract management and oversight teams were identified in the interagency agreements, key roles and responsibilities for the review and approval of invoices were not clearly defined. For example, the terms and conditions of the interagency agreement with GSA only vaguely described GSA's role in contract administration. However, the agreement did not specify the invoice review and approval steps to be performed. Likewise, the Mitretek contract provided a general description of its oversight duties, but did not specifically mention its responsibilities related to the invoice review and approval process. We did note, however, that FBI did not approve an invoice for payment until after it was notified by Mitretek that it had reviewed the invoice. Based on our discussions with the review team, Mitretek would review its own invoices before sending them forward to FBI for payment approval. Invoices Did Not Provide Adequate Support for All Charges: The failure to establish an effective review process was compounded by the fact that not all invoices provided detailed information required by the contracts and other information that would be needed to perform adequate reviews. Trilogy contractors were required to comply with various invoicing provisions of the FAR and the Trilogy contracts, including requirements to provide labor and various overhead rates, travel costs by trip, transaction detail for ODC, and purchase orders for equipment purchases. However, we found that the contractors, particularly CSC, often did not meet these requirements. For example: * CSC labor invoices did not include information related to individual labor rates or indicate which overhead rates were applicable to each employee--information needed to verify mathematical accuracy and to determine that the components of the labor charges were valid. * CSC invoices provided a summary of travel charges by category (airfare, lodging, etc.), but did not provide required information related to an individual traveler's trip costs. The travel invoices also did not provide cost detail by travel authorization number. Therefore, there was no way to determine that the trips billed were approved in advance or that costs incurred were proper and reasonable based on the location and length of travel. * CSC and SAIC invoices for ODC provided a summary of charges by category (shipping, office supplies, etc.); however, CSC did not provide required cost detail by transaction. In some cases, the category of charges was not even identified. For example, as shown in figure 3, within the ODC invoice, a subcategory entitled "other direct costs" made up $1.907 million of the $1.951 million invoice current billing total. No additional information was provided in the invoice to explain what made up these "other direct costs." Figure 3: Example of CSC ODC Invoice: [See PDF for image] [End of figure] * For purchased equipment, CSC invoices included a summary sheet-- indicating the total price billed, a brief description of items purchased, and the quantity of each item purchased--and a copy of the related "Bill of Material" (BOM).[Footnote 23] However, they did not individually identify each asset being billed by bar code, serial number, or some other method that would allow verification of assets billed to assets received. SAIC invoices also lacked the detailed information necessary to individually identify assets. This severely impeded FBI's ability to determine whether it had actually received the assets included on invoices and to subsequently track individual accountable assets on an item-by-item basis. We also found that Mitretek, a member of FBI's review team, submitted invoices that did not include detailed information needed to perform adequate reviews. For example, Mitretek's invoices did not include individual labor rates needed to verify rates charged with salary information or overhead rates needed to recalculate labor costs.[Footnote 24] As previously noted, Mitretek reviewed its own invoices before sending them forward to FBI for payment approval. Even though contractor invoices, particularly those from CSC, frequently lacked key information needed to review charges, we found through inquiries with the review team and the contractors that invoices were generally paid without requesting additional supporting documentation.[Footnote 25] Some Payments Made to Contractors Were for Questionable Costs: Because of the lack of fundamental internal controls over the process used to pay Trilogy invoices, FBI was highly vulnerable to payment of unallowable contractor charges. In an attempt to determine the validity of FBI's payments, we used forensic auditing techniques, including data mining and document analysis, to select certain contractor costs and requested supporting documentation from the contractors. We identified about $10.1 million of questionable contractor costs paid by FBI. These included payments for first-class travel and other excessive airfare costs, incorrect billings for overtime hours, potentially excessive labor rates, and other questionable subcontractor costs. The following sections provide additional information on the payments for questionable costs we found. Given FBI's poor control environment and the fact that we only reviewed selected FBI payments to Trilogy contractors that we identified with data mining and other forensic auditing techniques, other payments for questionable costs may have been made that have not been identified. First-Class Travel and Other Excessive Airfare Costs: During our review of CSC's supporting documentation for selected travel charges we found 19 first-class airline tickets purchased[Footnote 26] costing a total of $20,025, many of which exceeded the basic coach- class fares by significant margins.[Footnote 27] For example, in one case a traveler flew first class round trip between Providence, Rhode Island and San Francisco, California for $2,159. We estimated that a coach-class ticket for this same trip would have cost $1,119. In addition, 1 day after returning to Providence, this traveler flew back to San Francisco. The documentation provided by CSC did not explain or justify this first-class travel or unusual travel itinerary. The CSC contract called for airfare to be reimbursed to the extent allowable pursuant to the Joint Travel Regulations (JTR), which state that travelers must use basic economy or coach class unless the use of first- class travel is properly authorized and justified.[Footnote 28] Because the documentation provided by CSC for the 19 first-class tickets costing $20,025 that we identified did not contain authorizations or justifications, the cost of this travel in excess of a coach-class ticket is potentially unallowable. Table 2 provides specific examples of these potentially unallowable first-class travel costs. Table 2: Examples of CSC's Potentially Unallowable First-Class Travel: Itinerary: Chicago, IL to Pittsburgh, PA and back; Actual cost of first-class ticket: $926; Estimated cost of coach-class ticket[A]: $197; Percentage that first-class exceeded coach-class cost: 370%. Itinerary: One-way from Buffalo, NY to San Diego, CA; Actual cost of first-class ticket: $1,020; Estimated cost of coach-class ticket[A]: $295; Percentage that first-class exceeded coach-class cost: 246%. Itinerary: Wichita, KS to Washington, DC and back; Actual cost of first-class ticket: $1,984; Estimated cost of coach-class ticket[A]: $732; Percentage that first-class exceeded coach-class cost: 171%. Itinerary: One-way from Dallas/Fort Worth, TX to St. Louis, MO; Actual cost of first-class ticket: $518; Estimated cost of coach-class ticket[A]: $200; Percentage that first-class exceeded coach-class cost: 159%. Itinerary: One-way from Washington, DC to St. Louis, MO; Actual cost of first-class ticket: $723; Estimated cost of coach-class ticket[A]: $350; Percentage that first-class exceeded coach-class cost: 107%. Itinerary: Providence, RI to San Francisco, CA and back; Actual cost of first-class ticket: $2,159; Estimated cost of coach-class ticket[A]: $1,119; Percentage that first-class exceeded coach-class cost: 93%. Itinerary: One-way from Richmond, VA to Denver, CO; Actual cost of first-class ticket: $1,064; Estimated cost of coach-class ticket[A]: $566; Percentage that first-class exceeded coach-class cost: 88%. Itinerary: Tampa, FL to Washington, DC and back; Actual cost of first-class ticket: $836; Estimated cost of coach-class ticket[A]: $490; Percentage that first-class exceeded coach-class cost: 71%. Source: GAO analysis of supporting documentation provided by CSC. [A] Because historical costs for coach-class tickets were not available, we estimated the costs of coach-class tickets based on an average of current prices for a similar itinerary purchased 3 days in advance (which was CSC's average based on the trips we reviewed) and adjusted for inflation applicable to airfare. [End of table] During our review of FBI's payments for travel costs, we also identified 75 unusually expensive coach-class tickets that were purchased by the contractors for $100,847, which exceeded basic coach- class fares by approximately $49,848. Upon further inquiry with several airlines, we determined that most of these tickets were for "full fare" coach-class tickets. We noted that the airlines used most often by the contractors indicated that it is possible to obtain a free upgrade to first class with the purchase of the more expensive full-fare coach ticket. We found that in some instances, the current price of a full- fare coach ticket was higher than the current price of a first-class ticket. As discussed above, the JTR requires travelers to use basic economy or coach class unless the use of first-class travel is properly authorized and justified. The JTR defines economy class as basic accommodations that include a service level available to all passengers regardless of fare paid. Since full-fare coach tickets allow a traveler to upgrade to first class at no additional cost, full-fare coach class does not appear to be basic accommodations available to all passengers regardless of fare paid. As such, the purchase of full-fare coach-class tickets is a questionable cost. While the contracts incorporated the JTR, we determined that the JTR applies to civilian employees of the Department of Defense and is not considered appropriate "travel regulations" for contractors. The FAR, which would be appropriate for contractors, requires the use of the lowest customary standard, coach, or equivalent airfare[Footnote 29] and indicates that costs in excess of the lowest standard, coach, or equivalent airfare are unallowable. Had these provisions of the FAR been applied, the excessive cost of these tickets would have been potentially unallowable. We noted 62 full-fare coach tickets billed by CSC for $85,336, compared to an estimated cost of $41,978 for the basic fully refundable coach- class fares. We also identified 6 full-fare coach tickets billed by SAIC. In addition, we noted 5 trips billed by SAIC for subcontractor travel with excessive airfare costs for which the airfare class was not included in the supporting documentation provided by SAIC. Therefore, we could not determine whether these 5 trips were first class, full- fare coach, or some other class of travel that exceeded basic coach- class fares. These 11 tickets cost $11,610, compared to an estimated cost of $7,897 for the basic fully refundable coach-class fare. We further found 2 excessive airfare coach tickets billed by Mitretek that were upgraded to first class. These 2 tickets cost $3,901, compared to an estimated cost of $1,123 for the basic restricted coach-class fares.[Footnote 30] In total, the additional cost of $49,848 for the full-fare coach tickets and other excessive airfare are considered questionable. Table 3 provides examples of the excessive airfare travel costs of CSC, SAIC, and Mitretek. Table 3: Examples of Questionable Excessive Airfare Travel Costs: Contractor: Mitretek; Itinerary: Washington, DC to Phoenix, AZ and back; Ticket class: First-class upgrade[B]; Actual cost of ticket: $2,051; Estimated cost of basic coach-class ticket[A]: $480; Percentage that full-fare coach exceeded basic coach cost: 327%. Contractor: CSC; Itinerary: One-way from Los Angeles, CA to Philadelphia, PA; Ticket class: Full fare; Actual cost of ticket: $1,253; Estimated cost of basic coach-class ticket[A]: $307; Percentage that full-fare coach exceeded basic coach cost: 308%. Contractor: CSC; Itinerary: One-way from Las Vegas, NV to Washington, DC; Ticket class: Full fare; Actual cost of ticket: $1,171; Estimated cost of basic coach-class ticket[A]: $304; Percentage that full-fare coach exceeded basic coach cost: 285%. Contractor: CSC; Itinerary: One-way from San Francisco, CA to Cleveland, OH; Ticket class: Full fare; Actual cost of ticket: $1,049; Estimated cost of basic coach-class ticket[A]: $290; Percentage that full-fare coach exceeded basic coach cost: 262%. Contractor: Mitretek; Itinerary: Washington, DC to Portland, OR and back; Ticket class: First-class upgrade[B]; Actual cost of ticket: $1,850; Estimated cost of basic coach-class ticket[A]: $643; Percentage that full-fare coach exceeded basic coach cost: 188%. Contractor: CSC; Itinerary: One-way from San Diego, CA to Baltimore, MD; Ticket class: Full fare; Actual cost of ticket: $1,128; Estimated cost of basic coach-class ticket[A]: $413; Percentage that full-fare coach exceeded basic coach cost: 173%. Contractor: CSC; Itinerary: Atlanta, GA to Los Angeles, CA and back; Ticket class: Full fare; Actual cost of ticket: $2,121; Estimated cost of basic coach-class ticket[A]: $851; Percentage that full-fare coach exceeded basic coach cost: 149%. Contractor: CSC; Itinerary: Minneapolis/St. Paul, MN to Los Angeles, CA and back; Ticket class: Full fare; Actual cost of ticket: $2,107; Estimated cost of basic coach-class ticket[A]: $927; Percentage that full-fare coach exceeded basic coach cost: 127%. Contractor: CSC; Itinerary: One-way from Seattle, WA to Milwaukee, WI; Ticket class: Full fare; Actual cost of ticket: $1,038; Estimated cost of basic coach-class ticket[A]: $468; Percentage that full-fare coach exceeded basic coach cost: 122%. Contractor: CSC; Itinerary: Boston, MA to Los Angeles, CA and back; Ticket class: Full fare; Actual cost of ticket: $2,053; Estimated cost of basic coach-class ticket[A]: $1,141; Percentage that full-fare coach exceeded basic coach cost: 80%. Contractor: SAIC; Itinerary: Syracuse, NY to Washington, DC and back; Ticket class: Not determinable[C]; Actual cost of ticket: $862; Estimated cost of basic coach-class ticket[A]: $484; Percentage that full-fare coach exceeded basic coach cost: 78%. Contractor: CSC; Itinerary: Washington, DC to Los Angeles, CA and back; Ticket class: Full fare; Actual cost of ticket: $1,874; Estimated cost of basic coach-class ticket[A]: $1,090; Percentage that full-fare coach exceeded basic coach cost: 72%. Contractor: CSC; Itinerary: Washington, DC to San Francisco, CA and back; Ticket class: Full fare; Actual cost of ticket: $2,444; Estimated cost of basic coach-class ticket[A]: $1,490; Percentage that full-fare coach exceeded basic coach cost: 64%. Contractor: SAIC; Itinerary: Washington, DC to Chicago, IL; and back; Ticket class: Full fare; Actual cost of ticket: $942; Estimated cost of basic coach-class ticket[A]: $619; Percentage that full-fare coach exceeded basic coach cost: 52%. Contractor: SAIC; Itinerary: Denver, CO to Washington, DC; and back; Ticket class: Not determinable[C]; Actual cost of ticket: $1,570; Estimated cost of basic coach-class ticket[A]: $1,037; Percentage that full-fare coach exceeded basic coach cost: 51%. Source: GAO analysis of supporting documentation provided by contractors. [A] Because historical costs for coach-class tickets were not available, we estimated the costs of coach-class tickets based on an average of current prices for a similar itinerary purchased 3 days in advance (which was the average based on the trips we reviewed) and adjusted for inflation applicable to airfare. [B] The fare basis code for this ticket indicated that a first-class upgrade was obtained. We could not verify whether this ticket was purchased as a full-fare coach or some other class of travel that exceeded the basic coach-class fares. [C] We could not determine the airfare class of the ticket purchased because the supporting documentation provided did not include the fare basis code. [End of table] Excess Overtime Charges: During our review of labor charged by SAIC, we found that SAIC billed the Trilogy project for overtime hours worked by employees that exceeded the hours that would have been charged if SAIC followed the overtime policy informally agreed to by SAIC and FBI.[Footnote 31] Our calculations indicate that FBI may have overpaid an estimated $400,000 for these excess overtime charges. SAIC's task order, awarded in June 2001, stated that if work beyond the standard 40-hour work week was necessary to support the requirements of the task order, the government would not object to SAIC employees working an extended work week (EWW) (hours in excess of 40 per week). For designated EWW periods, exempt staff (professional staff normally not eligible for overtime compensation) would be paid a pro rata share (straight time) of their weekly salary based on the extended hours worked. EWW periods required SAIC management approval and were used when exempt staff were required to work extended hours for short periods of time due to special circumstances, such as accelerated project schedules or circumstances where employees could not dictate their work schedule. The first EWW period started August 31, 2002, and throughout the Trilogy project SAIC management approved 11 EWW periods for employees working on various Trilogy tasks. In March 2003, after the fourth EWW period started, SAIC implemented an EWW policy, agreed to with FBI, which decreased the amount of hours that would be billed to FBI. This policy stated that exempt staff would be compensated for hours worked that were greater than 90 hours in a 2-week pay period on an hour-for- hour basis. That meant that the first 10 hours of overtime would be uncompensated. In addition, a ceiling of 120 hours was established, meaning that employees would not be compensated for hours worked in excess of 120 in a pay period. SAIC agreed that it would not bill FBI for this uncompensated overtime. During our review of employee labor billings for the Trilogy project, we found that SAIC employees who charged EWW time after the March 2003 policy frequently charged for all hours worked beyond 80 in a pay period and that the cost of these hours was billed to and paid by FBI. We also noted some instances where employees charged EWW beyond the 120- hour ceiling per pay period, which were also billed. We discussed this issue with SAIC management and they agreed that their billing of EWW costs was not consistent with the policy that was established in March 2003 and indicated that they would research the issue further to determine whether corrections are necessary.[Footnote 32] Based on our review of the labor charges, it appears that FBI may have overpaid for more than 4,000 hours of EWW labor charges.[Footnote 33] Using average fully burdened labor rates for employees incorrectly billing EWW, we estimated that FBI may have overpaid EWW costs by approximately $400,000.[Footnote 34] Questionable Labor Rates: During our review of labor charged by CSC/DynCorp, we found that DynCorp Information Systems (DynIS), a subsidiary of DynCorp that billed about $42 million or 94 percent of DynCorp's direct labor, charged actual labor rates that may have exceeded rates that GSA asserts were established ceiling rates pursuant to the task order. CSC asserts that ceiling rates were never established. If ceiling rates were established, we estimated that FBI overpaid CSC by approximately $2.1 million. When DynCorp entered into the GSA FEDSIM Millennia contract, it agreed to ceiling rates that would be charged for its various labor categories, such as clerical and senior technician. The Millennia contract also stated that ceiling rates applicable to subcontractors would be negotiated separately for each task order awarded under the Millennia contract. After entering into the Millennia contract, DynCorp acquired a company that was renamed DynIS. Because DynIS' labor rates were not considered when DynCorp's ceiling rates were established under Millennia, DynCorp's proposal for the Trilogy task order listed DynIS as a subcontractor. In May 2001, GSA issued a Trilogy task order award document to DynCorp that had a section entitled "Ceiling Rates Applicable to DynIS" that included the following statement: "Ceilings are placed on all labor category and indirect rates used to establish the total cost for this task order—These ceiling rates are subject to negotiation pending the results of [Defense Contract Audit Agency] DCAA's[Footnote 35] audit." GSA officials told us they believed that DynIS labor category hourly rates in DynCorp's Trilogy proposal represented established labor category ceiling rates. GSA officials stated that they negotiated DynIS labor category ceiling rates with DynCorp.[Footnote 36] However, CSC stated that labor category ceiling rates were never established because they were never negotiated with GSA.[Footnote 37] In March 2003, CSC/DynCorp submitted and GSA approved a modification to the task order that, according to GSA, increased labor rates for several categories.[Footnote 38] However, CSC claims that this modification did not affect the ceiling rates because the ceilings were never established. Based on our review of DynCorp's labor invoices, we noted that several of DynIS' rates charged exceeded the labor rates that GSA contended were ceiling rates. For example, DynIS billed over 14,000 hours for work performed during 2001 for senior IT analysts working on the Trilogy project based on an average hourly rate of $106.14. However, if ceiling rates were established, the DynCorp proposal indicated that the Trilogy project would be charged a maximum of $68.73 per hour for a senior IT analyst working in the field or $96.24 per hour for a senior IT analyst working at headquarters. If ceiling rates were established, we estimated that FBI overpaid CSC/DynCorp by approximately $2.1 million for DynIS labor costs. [Footnote 39] Other Questionable Costs: We identified certain other payments to contractors that were for questionable costs. These costs were not supported by sufficient documentation to enable an objective third party to determine if each payment was a valid use of government funds.[Footnote 40] We further identified costs that were questionable as to whether they were necessary. Table 4 summarizes these questionable costs, which totaled about $7.5 million. Table 4: Other Questionable Costs Paid by FBI: Example: 1; Description of invoice charge: Subcontractor labor costs; Amount: $1,957,920. Example: 2; Description of invoice charge: Other direct costs - training; Amount: $4,746,045. Example: 3; Description of invoice charge: Other direct costs - equipment disposal; Amount: $762,262. Example: 4; Description of invoice charge: Subcontractor labor invoice - duplicate payment; Amount: $26,335. Total; Amount: $7,492,562. Source: GAO analysis of contractor and subcontractor invoices and supporting documentation. Note: Examples 2 and 3, combined, represent $5.5 million of inadequately supported CSC ODC. [End of table] A discussion of each of these questionable costs is provided below. Example 1--Subcontractor Labor Costs: CSC did not provide us adequate supporting documentation for almost $2 million of about $3.3 million of subcontractor labor charges we selected to review. The only documentation CSC could provide us for these charges were subcontractor invoices that lacked some of the basic information needed to assess the labor charges, such as the names of the subcontractor employees, hours billed, or individual labor rates. Therefore, CSC could not fully substantiate that the costs for services provided by the subcontractors that were charged to FBI's Trilogy project were appropriate. Example 2--Other Direct Costs/Training: CSC hired a subcontractor, CACI, to schedule and conduct training related to the Trilogy project. CACI billed more than $17 million ($13 million for labor and $4 million for facilities, equipment rentals, and other direct costs) to provide FBI agents and employees basic, intermediate, and advanced training in Microsoft Office applications, including Word, Excel, PowerPoint, and Outlook. FBI officials stated that FBI decided to conduct off-site, hands-on training for employees (instead of internal or CD-based training) because of the number of employees who had limited experience using computers and because FBI had insufficient space to set up training labs at their existing facilities. During our review of CSC ODC, we selected $4.7 million of these training charges from CACI and found that CSC was unable to provide us with adequate support for these charges. Subsequently, we requested supporting documentation from CACI for selected charges totaling about $3.5 million of these training costs. Our examination identified the following issues: * CACI could not adequately support almost $3 million that it paid to one event planning company. Since FBI decided to conduct their training off-site, CACI hired an event planner, which it paid almost $3.2 million to reserve hotel conference rooms, rent computer equipment for training sessions, and set up the conference rooms for the training. The bulk of the $3.2 million related to one purchase order for training at 72 sites over 3 months, which stated that costs could not exceed $2,992,526. This purchase order provided for payment of 50 percent of this amount to the event planner at the time the purchase order was issued (to cover costs that include prepayments for obtaining training facilities) and four equal monthly payments for the remaining balance. CACI provided us with the purchase order, which included a description of the services to be performed by the event planner. They also provided us copies of invoices from the event planner that included general descriptions of the services billed. CACI could not provide any further evidence of the actual costs of goods or services that were provided by the event planner, such as hotel invoices for the rental of conference rooms.[Footnote 41] CACI stated that documentation supporting actual costs of the event planner was not applicable because its agreement with the event planner was "fixed priced." CACI stated that the payment terms in the purchase order required only that CACI pay the event planner a series of payments in fixed amounts. However, CACI's assertion that supporting documentation of actual costs was not applicable was not supported by the terms of the purchase order, which included a related statement of work that specifically required documentation to support costs claimed by the event planner. According to the statement of work, the event planner was required to (1) provide data on actual costs incurred twice a month, (2) make every attempt to obtain the best pricing with respect to all costs, and (3) charge CACI only for services rendered, allowing for any cost savings from advance payments to be returned to CACI upon request.[Footnote 42] * CACI purchased about 30,000 ink pens and 30,000 highlighters for training sessions, at a cost of $19,705 and $32,314, respectively. The pens were custom made for the Trilogy training program. While there was supporting documentation for these costs and FBI officials stated that they preapproved the purchases as part of their acceptance of the Trilogy Pre-Training Education Plan, we question whether these purchases were necessary. Example 3--Other Direct Costs/Equipment Disposal: CSC was unable to provide us adequate supporting documentation for $762,262 in equipment disposal costs billed by two subcontractors. The documentation provided consisted of a spreadsheet that summarized costs of the subcontractors, but did not include receipts or other support to prove that these costs were actually incurred. Example 4--Subcontractor Labor Invoice-Duplicate Payment: Our review of SAIC's subcontractor labor charges found that FBI was billed twice for the same subcontractor invoice totaling $26,335. SAIC officials agreed that they double billed and stated that they would make a correction. Major Lapses in Accountability Resulted in Millions of Dollars of Missing Trilogy Equipment: FBI did not adequately maintain accountability for computer equipment purchased for the Trilogy project. FBI relied extensively on contractors to account for Trilogy assets while they were being purchased, warehoused, and installed. However, FBI did not establish controls to verify the accuracy and completeness of contractor records it was relying on, to ensure that only the items approved for purchase were acquired by the contractors, and to ensure that it received all those items acquired through its contractors. Moreover, once FBI took possession of the Trilogy equipment, it did not establish adequate physical control over the assets. Consequently, we found that FBI could not locate over 1,200 assets purchased with Trilogy funds, which we valued at approximately $7.6 million. In addition, during its physical inventory counts for fiscal years 2003 through 2005, FBI identified over 30 pieces of Trilogy equipment valued at about $167,000 that it reported as having been lost or stolen. Due to the significant weaknesses we identified in FBI's property controls, the actual amount of lost or stolen equipment could be even higher. FBI's Overreliance on Contractors Diminished Its Ability to Properly Account for Trilogy Assets: FBI relied on contractors to maintain records related to the purchasing, warehousing, and installation of about 62 percent of the equipment purchased for the Trilogy project.[Footnote 43] FBI's primary contractor responsible for delivering computer equipment to FBI sites was CSC. FBI officials told us they met regularly with CSC and its subcontractors to discuss FBI's equipment needs and a deployment strategy for the delivery of equipment. Based on these meetings, CSC instructed its subcontractors to purchase equipment, which was subsequently shipped to and put under the control of the subcontractors. Once equipment arrived at the subcontractors' warehouses, they were responsible for affixing bar codes on accountable items--all items valued above $1,000 and certain others considered sensitive that are required by FBI policy to be tracked individually. In addition, FBI directly purchased about $19.1 million of equipment for the Trilogy project that was shipped directly to CSC or its subcontractors. When equipment was shipped from subcontractor warehouses to FBI sites, the shipment included two CSC subcontractor-prepared reports. The first report, similar to a bill of lading, included all items shipped, including nonaccountable items such as cables. However, there was no requirement for FBI officials receiving the items to verify that the items included on this report were actually received. The second report listed accountable assets that were delivered such as desktop computers, scanners, printers, and network equipment that were available for installation at that location. This report was then used by the subcontractor during the installation of equipment at each FBI location to prepare the "Site Acceptance Listing" documenting equipment that had been accepted and installed at the site. At the completion of the site installation, both FBI and subcontractor officials were required to sign this Site Acceptance Listing. According to FBI headquarters officials, verification of the subcontractor-prepared Site Acceptance Listings represented a key control over Trilogy equipment, providing assurance that FBI received what it should have. However, based on our inquiries at two field offices we visited, we found that FBI officials who received equipment and signed the Site Acceptance Listing, may not have always verified the accuracy and completeness of these lists. An official from the Baltimore field office acknowledged that he signed these lists without verifying that the items included had actually been delivered and installed at his site. In addition, officials from the Newark field office said they felt comfortable that they had received all the items they were supposed to because of their close working relationship with the subcontractor who performed the installation; however, they acknowledged that they did not independently verify equipment included on the contractor lists that they had signed. FBI did not prepare its own independent lists of ordered, purchased, or paid-for assets, and therefore, it had no choice but to rely solely on the contractor lists to account for its Trilogy assets. Furthermore, when FBI received shipments from contractors, it did not compare purchasing and billing documentation to receiving documentation to verify that all items purchased were received as required by FBI's accountable asset manual. According to FBI policy, when shipments are received, a designated property custodian is responsible for ensuring that the items received are the same as those that were ordered and for determining whether a complete or partial shipment was received. However, FBI did not require that these procedures be followed for the Trilogy project because purchasing and billing documentation for the project was not site specific; instead, the program office instructed FBI staff to only verify the number of boxes received and not to open the boxes to verify the assets received until the deployment team arrived. In addition, FBI did not perform an overall reconciliation of total assets ordered and paid for to those received. Such a reconciliation would have been made difficult by the fact that invoices FBI received from CSC did not include item-specific information--such as bar codes, serial numbers, or shipping location. However, failure to perform such a reconciliation left FBI with no assurance that it had received all of the assets it paid for. FBI Lacked Adequate Physical Control over Trilogy Assets: Assets that were delivered to FBI sites by contractors were not entered into FBI's Property Management Application (PMA) in a timely manner, increasing the risk that assets could be lost or stolen without detection. FBI policy requires property management personnel to identify accountable items and enter them into PMA within 30 days of receipt. However, FBI officials acknowledged that Trilogy equipment had not been entered into PMA within 30 days, as required. We compared installation dates recorded in CSC's database[Footnote 44] of assets deployed to dates assets were recorded in PMA. As shown in table 5, we found that 71.6 percent of the CSC items that were recorded in PMA, representing 84 percent of the dollar value, were entered more than 30 days after receipt, contrary to FBI policy. In addition, 16.9 percent of the assets, representing 37 percent of the dollar value, were entered more than a year after receipt. When an asset is not recorded in the property system, there is no systematic means of identifying where it is located or when it is moved, transferred, or disposed of and no record of its existence when physical inventories are performed. This severely limits the effectiveness of the physical inventory in detecting missing assets. Table 5: Analysis of Time Taken by FBI to Enter CSC-Purchased Assets into PMA: Amount of time taken to enter records into PMA: 0-30 days; Percent of items tested: 28.4; Percent of dollar value tested: 16.3. Amount of time taken to enter records into PMA: 31-90 days; Percent of items tested: 27.5; Percent of dollar value tested: 19.2. Amount of time taken to enter records into PMA: 91-180 days; Percent of items tested: 10.1; Percent of dollar value tested: 9.1. Amount of time taken to enter records into PMA: 181-365 days; Percent of items tested: 17.1; Percent of dollar value tested: 18.5. Amount of time taken to enter records into PMA: 1-2 years; Percent of items tested: 9.3; Percent of dollar value tested: 12.9. Amount of time taken to enter records into PMA: 2-3 years; Percent of items tested: 6.9; Percent of dollar value tested: 22.3. Amount of time taken to enter records into PMA: 3-4 years; Percent of items tested: 0.7; Percent of dollar value tested: 1.7. Amount of time taken to enter records into PMA: Total; Percent of items tested: 100.0; Percent of dollar value tested: 100.0. Source: GAO analysis of PMA records and FBI's deployment schedule. [End of table] In an effort to identify the assets that should have been entered into PMA, FBI attempted to create, in 2005, an after-the-fact inventory listing of accountable and nonaccountable assets deployed. Because FBI had not prepared its own independent inventory listing of Trilogy assets ordered and paid for, it used the CSC-prepared list of equipment deployed as its basis to determine accountable assets.[Footnote 45] According to FBI, this list was supposed to include all CSC-deployed equipment that had been affixed with a bar code. However, FBI's ability to accurately identify accountable assets was hampered by its loss of control over bar codes. FBI policy identifies the use of bar codes as "the key control"[Footnote 46] for maintaining individual asset accountability and requires that bar codes be affixed to all accountable assets. Despite the importance of maintaining a reliable bar code system, FBI relied on contractors to affix the bar codes, but then did not track the bar code numbers given to contractors, the bar code numbers they used, or the bar code numbers returned. Moreover, FBI provided incorrect instructions to contractors, initially directing them to bar code certain types of nonaccountable computer pieces.[Footnote 47] An FBI official stated that when creating its after-the-fact listing of accountable and nonaccountable assets from the CSC listing, FBI tried to identify and list as nonaccountable those items that had been mistakenly bar coded. However, we found that FBI's accountable asset listing still included some nonaccountable assets that had been bar coded in error. Further, we noted that FBI's listing of nonaccountable assets incorrectly included some accountable items such as uninterruptible power supplies and network switches.[Footnote 48] As a result, FBI could not reliably determine the complete universe of Trilogy assets that should have been bar coded and designated as accountable property to be tracked separately by PMA. We also compared FBI's after-the-fact listing of accountable assets identified from the CSC-prepared listing to the asset records in FBI's PMA. We found that FBI's listing and or PMA included several errors and omissions in the listings, including: * accountable assets for which there was no listed bar code or serial number; * incorrect bar codes (for example, text bar codes or bar codes with too many digits); * items for which locations were listed as "unknown"; * assets with the same bar code with different serial numbers and/or locations; * incomplete and inaccurate asset descriptions; * items that matched to PMA by bar code but not by serial number; and: * items that matched to PMA by serial number but not by bar code. The FBI official who prepared the accountable asset listing said he gave this listing to each site with instructions to ensure that all of the assets had been entered into PMA in preparation for a 2005 physical inventory count. However, FBI did not follow up to determine whether all of the records in the inventory listing were actually entered into PMA. For site officials using the listing, the lack of complete and accurate information included in the inventory listing may have limited their ability to track some of the assets and ensure they were accounted for in PMA. FBI policy requires complete physical inventories of all accountable assets at least once every 2 years. Annually, a complete physical inventory of all accountable assets that are also capitalized assets (i.e., those with an acquisition cost of $25,000 or more) and "sensitive" property (e.g., laptop computers and weapons which are susceptible to theft) is performed. FBI's most recent biennial inventory of accountable assets occurred in the spring of 2005. To complete its inventory, FBI used scanner technology, directing employees responsible for performing the inventory to scan all items found at FBI locations that contained a bar code. PMA was updated to reflect the items that were located and scanned during the inventory and generated reports to identify new accountable assets that were not previously entered in the system.[Footnote 49] However, FBI did not compare the results of its inventory to its listing of accountable assets purchased under Trilogy to ensure that all of these assets were actually located during the inventory. Failing to perform this elemental step undermines the fundamental purpose of conducting physical inventories. FBI Is Unable to Locate Millions of Dollars of Trilogy Assets: Given that FBI did not ensure that all accountable Trilogy assets that should have been in its possession (i.e., those it paid for) were located during the physical inventory, we undertook several procedures in an attempt to do so. To perform this test work, we used FBI's inventory listing of CSC-purchased accountable equipment as well as similar FBI listings of assets FBI purchased directly (government furnished equipment or GFE) and that were purchased by SAIC. Although FBI's inventory listing of CSC-purchased accountable equipment included inaccurate and incomplete information, as previously discussed, we were able to reconcile the total number of items for selected types of equipment from its listing of accountable CSC-purchased equipment to the number of these assets invoiced by CSC. This provided some assurance that the listing of accountable CSC-deployed equipment purchased by both CSC and FBI for those asset types includes all accountable assets FBI paid for and that should be in FBI's possession. This was done for selected CSC-purchased accountable assets,[Footnote 50] which represented approximately 76 percent of the total number of CSC-purchased equipment, and all SAIC-purchased assets. Therefore, we used these asset listings to determine whether accountable assets were located during FBI's most recent physical inventory. We obtained several iterations of PMA listings and inventory reports from FBI and attempted to trace the assets to these reports. Collectively, these listings and reports should have included all accountable Trilogy assets in FBI's possession at the time of its 2005 inventory. Based on this comparison, we identified 1,205 accountable Trilogy assets, with an estimated value of approximately $7.6 million that FBI has been unable to locate or otherwise account for. We estimated this value using the lowest per-unit-cost based on the Trilogy equipment-pricing sheets that were prepared by FBI and used in recording the cost of the same types of assets in PMA. If we could not identify a price for a certain type of accountable asset in FBI's equipment-pricing sheets, we identified the lowest price on the accountable and capitalized assets spreadsheet prepared by FBI's finance division. When the cost was not available on either of these documents, or when the item was unknown, we did not attempt to estimate the asset's value. As a result, our estimated value of lost or stolen equipment does not include 103 of the 926 CSC-purchased items we identified, such as Paradyne frame savers and Optical HBA drivers, and therefore is understated. Table 6 provides a description and estimated value for the assets for which we could identify unit cost. Table 6: Trilogy-Purchased Items Not Located by FBI: CSC-purchased items: Item description: Data storage equipment; Number of items: 35; Total estimated value: $2,404,853. Item description: Network equipment: Switches; Number of items: 269; Total estimated value: $1,504,624. Item description: Network equipment: Routers; Number of items: 147; Total estimated value: $367,151. Item description: Network equipment: Servers; Number of items: 21; Total estimated value: $192,245. Item description: Network analysis equipment[A]; Number of items: 33; Total estimated value: $228,325. Item description: Uninterrupted power source[B]; Number of items: 120; Total estimated value: $176,812. Item description: CPUs/Computer desktops; Number of items: 113; Total estimated value: $129,350. Item description: Printers; Number of items: 58; Total estimated value: $99,231. Item description: Miscellaneous equipment (e.g., laptops, flat panel monitors, etc.); Number of items: 130; Total estimated value: $170,795. Item description: CSC total; Number of items: 926; Total estimated value: $5,273,386. SAIC and FBI-purchased UAC items: Item description: Network equipment: Servers; Number of items: 2; Total estimated value: $90,597. Item description: Network equipment: Routers; Number of items: 2; Total estimated value: $28,000. Item description: Network equipment: Switches; Number of items: 3; Total estimated value: $160,355. Item description: Data storage equipment; Number of items: 5; Total estimated value: $1,012,666. Item description: Load balancers; Number of items: 3; Total estimated value: $281,601. Item description: Printers and scanners; Number of items: 224; Total estimated value: $398,885. Item description: Miscellaneous equipment (encryption equipment and CPUs); Number of items: 3; Total estimated value: $30,869. Item description: SAIC and FBI total; Number of items: 242; Total estimated value: $2,002,973. FBI-purchased IPC/TNC GFE: Item description: Network equipment: Servers; Number of items: 28; Total estimated value: $314,668. Item description: CPUs/Computer desktops; Number of items: 1; Total estimated value: $2,850. Item description: Laptops; Number of items: 7; Total estimated value: $12,190. Item description: Monitor; Number of items: 1; Total estimated value: $1,075. Item description: FBI total; Number of items: 37; Total estimated value: $330,783. Item description: Total; Number of items: 1,205; Total estimated value: $7,607,142. Source: GAO analysis of FBI listings of accountable assets and PMA reports. [A] This equipment includes electronic testing tools used to ensure proper installation of network equipment. [B] An uninterrupted power source is a constantly charging battery pack which powers the computer. [End of table] As of November 30, 2005, FBI was unable to sufficiently explain why these items were not accounted for in PMA and/or could not provide adequate documentation that the assets had been located. An FBI official stated that some of the assets included in the listing of CSC- purchased equipment would not be expected to be in PMA because some were replaced. For example, according to the official, some of the CSC- purchased switches were replaced due to a heating malfunction. However, FBI did not provide us with documentation related to replaced items, and therefore we could not determine which units if any were replaced and/or which units should still be on hand. The FBI official also told us that, even though he attempted to remove all nonaccountable items from the listing of CSC-purchased equipment, some nonaccountable items may still have been included. For example, FBI told us some purchased components that were a part of an accountable asset unit may have been bar coded even though the item by itself was not an accountable item. Using FBI guidance on accountable property,[Footnote 51] we determined that 103, or about 11.1 percent, of the missing 926 CSC-purchased assets may represent nonaccountable units. Because FBI was unable to provide us with location information for these items, we could not definitively determine whether they represent nonaccountable components or are separate accountable assets that were not in PMA and could not be located. FBI had no further explanation for why it could not locate the missing assets we identified or whether the missing assets we identified may expose confidential and sensitive information and data to unauthorized users. In addition to the missing items discussed above, FBI could not initially locate another 25 purchased assets--highly-sensitive encryption equipment--in its PMA system. Subsequently, FBI officials were able to provide the bar codes, locate the encryption equipment,[Footnote 52] and provide evidence that all of the items were now in its PMA system.[Footnote 53] The officials stated the equipment was not originally required to be bar coded or tracked in PMA, but that it was tracked several different ways by serial number. The officials also explained that the problem resulted mostly from FBI modifications to the equipment that required revisions to the serial numbers listed in the invoices. Regardless of the fact that the equipment was subsequently located after research and inquiries, such highly sensitive equipment needs to be properly and timely accounted for to ensure the precise location of the equipment can be immediately determined at all times. In addition to the items we found missing, FBI's property management division reported 37 CSC-purchased Trilogy assets, totaling approximately $167,000, that were determined to be lost or stolen during its physical inventory counts for fiscal years 2003 through 2005.[Footnote 54] The assets reported as lost or stolen included computers and servers, which may have contained sensitive and confidential information. According to FBI policy, for items in PMA that cannot be located during the inventory, a "Report of Lost or Stolen Property" must be submitted to FBI headquarters. Due to security concerns, FBI did not provide us copies of these reports for the property items that were not located during the 2003, 2004, and 2005 inventories. Therefore, it is unclear what type of security risk if any these lost/stolen assets represent. Conclusions: FBI's Trilogy IT project spanned 4 years and the reported costs exceeded $500 million. Our review disclosed that there were serious internal control weaknesses over the process used by FBI and GSA to approve contractor charges related to Trilogy, which made up the vast majority of the total reported project cost. While our review focused specifically on the Trilogy program, the significance of the issues identified during our review may be indicative of more systemic contract and financial management problems at FBI and GSA, in particular when using cost-reimbursable type contracts and interagency contracting vehicles. These weaknesses resulted in the payment of millions of dollars of questionable contractor costs, which may have unnecessarily increased the overall cost of the project. Unless FBI strengthens its controls over contractor payments, its ability to properly control the costs of future projects involving contractors, including its new Sentinel project, will be seriously compromised. Additionally, to the extent that GSA enters into similar interagency agreements, it will continue to be exposed to oversight lapses until it reassesses its procedures. Further, weaknesses in FBI's controls over the equipment acquired for Trilogy resulted in millions of dollars in missing equipment, and call into question FBI's ability to adequately safeguard its equipment, as well as confidential and sensitive information that could be accessed through that equipment from unauthorized use. Recommendations for Executive Action: We are making the following 27 recommendations to the Director of FBI and the Administrator of General Services to (1) facilitate the effective management of interagency contracting, (2) mitigate the risks of paying unallowable costs in connection with cost-reimbursement type contracts, and (3) improve FBI's accountability for and safeguarding of its computer equipment. To improve FBI's controls over its review and approval process for cost- reimbursement type contract invoices, we recommend that the Director of FBI instruct the Chief Financial Officer to establish policies and procedures so that: * Future interagency agreements establish clear and well-defined roles and responsibilities for all parties included in the contract administration process, including those involved in the invoice review process, such as contracting officers, technical point of contacts, contracting officer's technical representatives, and contractor personnel with oversight and administrative roles. * Appropriate steps are taken during the invoice review and approval process for every invoice cost category (i.e., labor, travel, other direct costs, equipment, etc.) to verify that the (1) invoices provide the information required in the contract to support the charges, (2) goods and services billed on invoices have been received, and (3) amounts are appropriate and in accordance with contract terms. * The resolution of any questionable or unsupported charges on contractor invoices identified during the review process is properly documented. * Labor rates, ceiling limits, treatment of overtime hours, and other key terms for cost determination are clearly specified and documented for all contracts, task orders, and related agreements. * Future contracts clearly reflect the appropriate Federal Acquisition Regulation travel cost requirements, including the purchase of the lowest standard, coach, or equivalent airfare. * An appropriate process is in place to assess the adequacy of contractor's review and documentation of submitted subcontractor charges before such charges are paid by FBI. In light of the findings in this report, we recommend that the Administrator of General Services instruct the director of FEDSIM to reassess its procedures in connection with (1) interagency contracts and (2) delegated contract administration responsibilities, including the following: * Clearly defining the roles and responsibilities of each party in interagency agreements, and particularly those related to reviewing and approving invoices. * Assessing the adequacy of its invoice review and approval polices, including specific steps to be performed by each party so that (1) invoices provide the information required in the contract to support the charges, (2) goods and services billed on invoices have been received, (3) amounts are appropriate and in accordance with contract terms, and (4) the resolution of any questionable or unsupported charges on contractor invoices identified during the review process is clearly documented. * Clearly documenting labor rates, ceiling limits, treatment of overtime hours, and other key terms for cost determination for all contracts, task orders, and related agreements. * Clearly reflecting in future contracts the appropriate Federal Acquisition Regulation travel cost requirements, including the purchase of the lowest standard, coach, or equivalent airfare. * Confirming that contractors properly review and support submitted subcontractor charges. To address issues on the Trilogy project that could represent opportunities for recovery of costs, we recommend that the Administrator of General Services, in coordination with the Director of FBI: * Confirm SAIC's informal Extended Work Week policy and work with SAIC to determine and resolve any overpaid amounts. * Further investigate whether DynIS' labor rates exceeded ceiling rates and pursue recovery of any amounts determined to have been overpaid. * Determine whether other contractor costs identified as questionable in this report should be reimbursed to FBI by contractors. * Consider engaging an independent third party to conduct follow-up audit work on contractor billings, particularly areas of vulnerability identified in this report. To improve FBI's accountability for purchased assets, we recommend that the Director of FBI instruct the Chief Financial Officer to: * Establish policies and procedures so that (1) purchase orders are sufficiently detailed so that they can be used to verify receipt of equipment at FBI sites, and (2) contractor invoices are formatted to tie directly to purchase orders and facilitate easy identification of equipment received at each FBI site. * Reinforce existing policies and procedures so that when assets are delivered to FBI sites, they are verified against purchase orders and receiving reports. Copies of these documents should be forwarded to FBI officials responsible for reviewing invoices as support for payment. * Establish policies and procedures so that invoices are paid only after all verified purchase order and receipt documentation has been received by FBI payment officials and reconciled to the invoice package. * Establish a policy to require that, upon receipt of property at FBI sites, FBI personnel immediately identify all accountable assets and affix bar codes to them. * Revise FBI's policies and procedures to require that all bar codes are centrally issued and tracked through periodic reconciliation of bar codes issued to those used and remaining available. Assigned bar codes should also be noted on a copy of the receiving report and forwarded to FBI's Property Management Unit. * Revise FBI policies and procedures to require that accountable assets be entered into PMA immediately upon receipt rather than within the current 30-day time frame. * Require officials inputting data into PMA to enter (1) the actual purchase order number related to each accountable equipment item bought, (2) asset descriptions that are consistent with the purchase order description, and (3) the physical location of the property. * Establish policies and procedures related to the documentation of rejected or returned equipment so that the (1) equipment that is rejected immediately upon delivery is notated on the receiving report that is forwarded to FBI officials responsible for invoice payment; and (2) equipment that is returned after being accepted at an FBI site (e.g., items returned due to defect), is annotated in PMA, including the serial number and location of any replacement equipment, under the appropriate purchase order number. * Reassess overall physical inventory procedures so that all accountable assets are properly inventoried and captured in the PMA system and that all unlocated assets are promptly investigated. * Expand the next planned physical inventory to include steps to verify the accuracy of asset identification information included in PMA. * Establish an internal review mechanism to periodically spot check whether the steps listed above--including verifications of purchase orders and receiving reports against received equipment, immediate identification and bar coding of accountable assets, maintenance of accurate asset listings, prompt entry of assets into PMA, documentation of rejected and returned equipment, and improved bar coding and inventory procedures--are being carried out. * Investigate all missing, lost, and stolen assets identified in this report to (1) determine whether any confidential or sensitive information and data may be exposed to unauthorized users; and (2) identify any patterns related to the equipment (e.g., by location, property custodian, etc.) that necessitates a change in FBI policies and procedures, such as assignment of new property custodians or additional training. Agency Comments and Our Evaluation: In written comments reprinted in appendix III, FBI stated that it concurred with our recommendations and that it has made and continues to make significant structural and procedural changes to address our recommendations, taking critical steps to strengthen internal controls. FBI also provided additional information related to Trilogy assets we identified as missing. In written comments reprinted in appendix IV, GSA stated that it accepted our recommendations, did not believe that 1 of them was needed, and described some of the improvements to its internal controls and other business process changes already implemented. GSA also expressed concern with some of our observations and conclusions related to the invoice review and approval process and our analysis of airfare costs. FBI and GSA also provided technical comments, which we have incorporated as appropriate. In its comments, FBI stated that executive management at FBI has directed a sustained effort to address and correct weaknesses identified in our report and other Trilogy reviews. FBI further stated that attention is being focused on four areas: (1) audit capability, (2) property management, (3) contracting services, and (4) IT investments. If properly implemented, the activities outlined in FBI's letter should help improve FBI accountability for future IT acquisitions and other contract services. In this regard, vigilant oversight will be needed to ensure controls are correctly designed and operating effectively to protect assets and prevent improper payments. Further, in its comments, FBI stated that more than 44,000 pieces of accountable property were successfully deployed and tracked in the FBI's PMA during the Trilogy project. FBI also stated that the 1,404 items we initially reported as missing or improperly documented represented approximately 3 percent of the accountable assets. We question both of these statements. Because of the control weaknesses discussed in our report, FBI does not have a reliable basis to know the number of Trilogy assets it purchased or how many should have been tracked as accountable assets. Further, since we did not test all the assets purchased, more may be missing. FBI also stated that as of January 2006, it had accounted for more than 1,000 of the 1,404 items we reported as missing or improperly documented. During our agency comment period, FBI indicated that it found 237 items we previously identified as missing and provided evidence, not made available during our audit, to sufficiently account for 199 of these items. We adjusted the missing assets listing in our report to reflect 1,205 (1,404 - 199) assets as still missing. In February 2006, FBI informed us that the approximately 800 remaining items noted in its official agency response included (1) accountable assets not in PMA because they were either incorrectly identified as nonaccountable assets or mistakenly omitted, (2) defective accountable assets that were never recorded in PMA and subsequently replaced, and (3) nonaccountable assets or components of accountable assets that were incorrectly bar coded. We considered these same issues during our audit and attempted to determine their impact. For example, as stated in our report, FBI told us that components of some nonaccountable assets that were part of a larger accountable item may have been mistakenly bar coded. Using FBI guidance on accountable property, we determined that 103 or about 11 percent of the 926 missing assets purchased by CSC may have represented nonaccountable components. Because FBI could not provide us with the location information, we could not definitively determine whether the items were accountable assets or not. During the course of our audit, FBI was not able to provide us with any evidence to support their other statements regarding the reasons the assets could not be located. While we are encouraged by FBI's current efforts to account for these assets, its ability to definitively determine their existence has been compromised by the numerous control weaknesses identified in our report. Further, the fact that assets have not been properly accounted for to date means that they have been at risk of loss or misappropriation without detection since being delivered to FBI--in some cases for several years. While GSA said it accepted all of our recommendations, it expressed reservations regarding our recommendation that GSA should clearly reflect appropriate FAR travel cost requirements in future contracts. GSA stated in its comment letter that it believed that the requirements outlined in the applicable FAR section 31.205-46 and stipulated in the task orders were more than adequate. In a subsequent conversation, we asked the GSA contracting officer why the language in the CSC and SAIC task orders and the Mitretek contract, which stated that long-distance travel would be reimbursed to the extent allowable pursuant to the JTR, was considered appropriate by GSA. The GSA contracting officer stated that, while not specified in the contract language, the reference to the JTR related only to per diem rates and allowances when determining the reasonableness of the travel costs, such as lodging and mileage reimbursements. She further stated that the FAR would apply to all other travel reimbursement determinations. We do not agree that our recommendation is unnecessary. In our view, the references to the JTR create ambiguity. The FAR cost allowability clause 52.216-7 states that when determining allowability, in addition to FAR cost principles, the terms of the contract also apply. Therefore, the reference to the allowability under the JTR could have caused confusion with the contractors regarding what long-distance travel costs were allowed, including airfare costs. We continue to believe that the task orders should have more clearly described the applicable travel requirements. Regarding the invoice review and approval process, GSA stated that each member of the review team--FBI, GSA, and Mitretek--played a unique and mutually understood role. In particular, GSA stated that Mitretek's role in the invoice review and approval process was significant and that it was reasonable for GSA to have relied on input from FBI, via Mitretek, in approving invoices for payment. GSA also referred to procedures to preapprove ODC and equipment purchases. Further, GSA stated that it believed that the procedures to process invoices were generally sound and that contractors are required to maintain records to adequately demonstrate that costs claimed have been incurred, and are reasonable, allowable, and allocable. GSA also stated that it will have DCAA audit the contract costs to determine if any costs are unallowable, unreasonable, or unallocable and will use the audit results as a basis to pursue remedies to recoup funds and assess penalties as may be applicable. We disagree with GSA regarding review team roles and the review process. Based on discussions with members of the review team, our review of supporting documentation, and our assessment of the outcomes of the review process, it is clear that the invoice review and approval process was inadequate. The roles and responsibilities of the review team members were not clearly defined or documented and this led to confusion among the review team members about each member's role. Regarding Mitretek's role, Mitretek officials stated that they performed a limited review of only labor invoices. Before relying on others, GSA should have verified its understanding of each member's roles and responsibilities and confirmed that the appropriate functions were being performed. In addition, while there were procedures to preapprove ODC and equipment purchases, the review team did not effectively link the preapproval and the invoice review and approval processes, especially in relation to CSC, in part because CSC invoices lacked detailed information needed to verify that charges billed were in fact preapproved. Further, while contractors are required to maintain records to adequately support costs, we found that the review team generally did not request additional documentation such as travel vouchers or subcontractor invoices to support amounts billed. If the review team had a systematic process in place to review costs, it may have questioned some of the excessive airfare we identified and found, as we did, that CSC lacked documentation to adequately support subcontractor charges. It is a management function and sound business practice to have a process in place to ensure that contractors have such documentation. Having such processes and questioning amounts billed would also allow for corrective measures to be implemented as, and if, problems were found. In addition, while we agree that a DCAA audit of contract costs can provide a detective control to help determine whether contractor costs were proper, reliance on an after-the-fact audit is not an acceptable replacement for the type of real-time monitoring and oversight of contractor costs--preventative controls-- we recommend in this report. Further, a DCAA audit of civilian contractor costs is not automatic and would require an additional cost to the government to procure. The review team largely operated in an environment of trust without an adequate basis for knowing whether the contractor billings were reasonable and costs claimed were allowable. Effective internal control calls for a sound, on-going invoice review and approval process as the first line of defense in preventing unallowable costs. Regarding our analysis of travel costs, GSA stated that our conclusions did not account for the ever-changing travel schedules and itineraries necessitated by changes in FBI requirements. GSA also stated that a hypothetical standard coach-class ticket does not provide a benchmark to make a valid price comparison, even if adjusted for inflation, because the airline travel industry has had significant changes with respect to pricing of airline tickets. GSA also stated that they believe it is impossible at this date to look back over 5 years and estimate what may have been a reasonable airfare price. We disagree. Our analysis and conclusions related to travel did take into account the possible conditions that could justify airfare costs in excess of the lowest customary coach-class fare. The FAR requires supporting documentation for first-class and other excessive airfare costs of the nature we identified to justify the higher airfare costs. No such documentation was provided to us to justify the excessive costs we identified. To estimate the cost of the coach-class tickets, we assumed that tickets were purchased 3 days in advance (which was the average based on the trips we reviewed) and did not include a Saturday night stay over. Specifically, we (1) used the Web sites of the airlines used by each traveler, (2) searched for standard fully- refundable[Footnote 55] coach-class tickets with the same destinations, (3) calculated an average cost based on the lowest and highest ticket prices available at the time of our search, and (4) adjusted the average cost for inflation applicable to airfare. We believe that this approach, which closely approximated what travelers were doing at that time, resulted in reasonable estimates as to how much the travel should have cost. We also believe that adjusting current fares for inflation applicable to airfare results in a reasonable benchmark to compare to historical prices, since it does take into account price changes as a result of changes in the airline industry, including the effects of competition. Lastly, we fully agree with GSA that the passage of time makes it difficult to determine historical airfare costs, which is another reason that costs should be reviewed real time instead of as part of an after-the-fact audit. An after-the-fact analysis is no substitute for the contemporaneous monitoring and oversight that we recommend in this report. More specific discussions are provided following GSA's comments, which are reprinted in appendix IV. As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its date. At that time, we will send copies of this report to the Director of the FBI, the Acting Administrator of GSA, and interested congressional committees. Copies will also be made available to others upon request. In addition, the report will be available at no charge on the GAO Web site at [Hyperlink, http://www.gao.gov]. If you or your staffs have any questions about this report, please contact me at (202) 512-9508 or [Hyperlink, calboml@gao.gov]. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributors to this report are acknowledged in appendix V. Signed by: Linda M. Calbom: Director, Financial Management and Assurance: Appendixes: Appendix I: Key Trilogy Milestones: The Federal Bureau of Investigation's (FBI) Trilogy project experienced several delays, as shown in figure 4. Figure 4: Key Trilogy Milestones: [See PDF for image] [End of figure] [End of section] Appendix II: Scope and Methodology: To determine whether the Federal Bureau of Investigation's (FBI) internal controls provided reasonable assurance that improper payments to contractors would not be made or would be detected in the normal course of business, we used the Standards for Internal Control in the Federal Government, Guide for Evaluating and Testing Controls Over Sensitive Payments, and The Executive Guide on Strategies to Manage Improper Payments: Learning from Public and Private Sector Organizations as a basis for assessing FBI's internal control structure over its Trilogy program. We also reviewed our prior reports, as well as those by the Department of Justice's Office of Inspector General on Trilogy issues; Trilogy contracts and interagency agreements; and contractor invoices and other documentation supporting goods provided and services rendered. In addition, we conducted interviews with officials from FBI, the General Services Administration, the Department of the Interior, and the contractors, and performed walkthroughs to gain an understanding of the processes used to review and approve invoices. Validity of Payments: To determine whether FBI's payments to contractors were properly supported as a valid use of government funds, we performed data mining, document analysis, and other forensic auditing techniques to select transactions to test. We reviewed documentation maintained by the review team, contractors, or subcontractors to assess the allowability of costs based on Trilogy contract documents and applicable federal regulations, such as the Federal Acquisition Regulation, Federal Travel Regulation, and Joint Travel Regulations. While we identified some payments for questionable costs, our work was not designed to identify all questionable payments or to estimate their extent. The following provides more details on our testing of payments to FBI's contractors- -Science Applications International Corporation (SAIC), Computer Sciences Corporation (CSC), and Mitretek--for labor, subcontractor labor, travel, and other direct costs (ODC). Payments to SAIC: * To test payments for labor costs, we obtained from SAIC a database of hours charged to the Trilogy project by employee and pay period. Using this database and labor invoice detail, we selected 21 employees based on either (1) a high number of hours worked, (2) a high dollar amount billed, or (3) billing in more than one labor category. For these 21 employees, to test the labor rates billed, we compared rates billed to salary information. In addition, for subsets of these 21 employees, we compared the hours billed to hours reported in SAIC's labor database and tested the mathematical accuracy of the labor costs billed. To determine the reasonableness of extended work week (EWW) hours charged to the Trilogy project, using the database we analyzed the total, EWW, and uncompensated hours charged by employee. We also compared the average fully burdened labor rates charged to ceiling rates to determine whether the rates were below the ceilings. * To test payments for subcontractor labor costs, we obtained from SAIC a database of all subcontractor labor charges. In order to determine whether the database was complete, we verified that the database reconciled with SAIC's subcontractor billings. We then selected subcontractor invoices to review based on a high dollar amount billed or unusual billing patterns. We analyzed supporting documentation such as subcontractor invoices and time sheets from SAIC for about $17.2 million, or 37 percent, of payments for SAIC subcontractor labor. * To test payments for travel costs, using detail included in SAIC's travel invoices and copies of travel authorizations provided by SAIC, we selected transactions to review based on (1) high airfare costs, (2) actual costs that exceeded authorized amounts, and (3) unusual billing patterns. We analyzed supporting documentation, such as travel vouchers, receipts, and subcontractor invoices, from SAIC for about $154,000, or 45 percent, of payments for SAIC travel costs. * To test payments for ODC, using detail included in SAIC's ODC invoices, we selected transactions with unusually large amounts within a category or with an unusual category description. We analyzed supporting documentation, such as invoices or other documentation, from SAIC for about $307,000, or 61 percent, of payments for SAIC ODC. Payments to CSC: Because CSC was unable to readily provide us transaction-level detail for all labor, travel, and ODC charges, we selected 11 invoices based on the amounts billed and the time periods covered. CSC was able to provide us transaction-level detail for these 11 invoices, which represented $14.7 million or about 33 percent of labor costs;[Footnote 56] $3.1 million or about 33 percent of travel costs; and $2.4 million or about 27 percent of ODC charges. Using these 11 invoices as our data source we performed the following tests of CSC labor, travel, and ODC. * We recalculated the total labor charged for three labor categories in 7 of the 11 invoices to verify that the invoice amounts were calculated correctly.[Footnote 57] We also selected 11 employees based on either (1) high number of hours worked, (2) a high dollar amount billed, or (3) billing in more than one labor category. For these 11 employees, we compared the hours billed to time sheets and verified hourly rates by reviewing each employee's salary history. In total, the 11 selected employees billed around $850,000 on the 11 invoices we reviewed. We tested the reliability of the detail provided by comparing the hours and amounts to labor invoices. We compared the average fully burdened rates charged to ceiling rates. * We selected travel charges that were high in amount or exhibited unusual billing patterns. We reviewed travel vouchers for these selected charges. Because we identified possible first-class and unusual coach-class travel in these selections, we obtained and reviewed additional supporting documentation for CSC-purchased airline tickets beyond the initial 11 invoices selected for review. * We selected ODC transactions with unusually large amounts within a category or in an unusual category (such as computer hardware). Because of anomalies we identified in our initial review, we selected additional transactions to review beyond the initial 11 invoices. In total, we analyzed supporting documentation for about $7.0 million, or about 80 percent, of payments for CSC ODC during the Trilogy project. * To test payments for subcontractor labor costs, we obtained from CSC transaction-level detail for 12 of its subcontractors during the Trilogy project. From the transaction-level detail, we selected charges to review based on (1) high number of hours worked, (2) a high amount billed, and (3) other unusual billing patterns. We obtained and analyzed supporting documentation, such as subcontractor invoices, from CSC for about $3.3 million, or 4 percent, of the $75 million charged by CSC as subcontractor labor costs during the Trilogy project.[Footnote 58] Payments to Mitretek: * To test payments for labor costs, we obtained transaction detail for three labor invoices, which represented $1.5 million or 8 percent of the payments for Mitretek labor. We tested the mechanical accuracy of the invoice calculation and selected one of the invoices and verified hours billed compared to time cards and hourly rates charged compared to salary histories. * To test payments for travel costs, we obtained and analyzed the supporting documentation, such as travel vouchers, for all travel costs on two invoices. These invoices represented $11,211 or about 13 percent of payments to Mitretek for travel costs. * To test payments for ODC,[Footnote 59] we obtained and analyzed the supporting documentation, such as invoices and receipts, for all ODC costs on two invoices. These invoices represented $139,083 or about 8 percent of payments to Mitretek for ODC. FBI's Asset Accountability: To determine whether FBI maintained proper accountability for assets purchased with Trilogy project funds, we used our Standards for Internal Control as a basis to assess FBI's control structure over its Trilogy assets. We interviewed FBI, contractor, and subcontractor staff to identify and assess the controls in place over the ordering, purchasing, and receipt of Trilogy equipment. The following provides more details on our testing of Trilogy equipment purchased for FBI by CSC and SAIC, or directly by FBI: * To determine whether FBI approved for purchase all assets acquired for the Trilogy project, we obtained FBI consents to purchase, Bills of Material, and invoices and compared the total assets approved to be purchased to assets actually purchased. * To determine whether FBI Trilogy accountable assets listed in PMA were recorded in a timely manner, we obtained documentation from FBI and contractors for accountable assets purchased by CSC that identified the bar codes assigned to accountable assets and the date the equipment was received by FBI. We did not perform this test for SAIC-purchased assets because the assets represented only .8 percent of the total assets purchased with Trilogy funds. We also did not perform this test for FBI direct purchases since the supporting documentation did not provide bar codes or serial numbers for individual assets. We compared the bar codes on the listings to FBI's Property Management Application (PMA) which included the date the asset was entered into PMA. * To assess the accuracy and completeness of the FBI-prepared listings of CSC-and SAIC-purchased assets, we (1) analyzed the listings to identify any irregularities such as duplicate bar codes or missing information; (2) obtained the CSC equipment invoices and compared the total number of pieces billed on the CSC invoices for four selected accountable asset types that represented about 76 percent of the total CSC assets purchased to FBI's listing; and (3) obtained the SAIC listings of Trilogy equipment returned to FBI, SAIC's equipment invoices, and FBI's listing of VCF assets and compared for each item the amount of equipment per the invoices to the SAIC listing and then to FBI's VCF listing. * To determine whether FBI had in its possession all accountable assets purchased for it by CSC and SAIC, we compared the complete listing of bar codes from FBI's VCF and CSC listings to PMA to identify any bar codes not recorded in PMA. * To test the accuracy of the data included in the PMA accountable asset records, we compared the data for each accountable asset, such as bar code number, serial number, asset description, and asset location, to FBI's listing and followed up on any discrepancies. * To identify Trilogy assets that had been reported as lost or stolen by FBI, we obtained a listing of all assets identified as lost or stolen by FBI during its annual inventories for years 2003, 2004, and 2005. We then compared this listing, by bar code, to FBI's CSC and VCF equipment listings to determine which of these assets had been acquired for the Trilogy project. The scope of our review covered all assets purchased from the inception of the Trilogy contracts (May 2001) through December 2004 and included Trilogy assets that were either purchased directly by FBI or by one of the two primary Trilogy contractors, CSC and SAIC. We provided FBI a draft of this report and GSA a draft of applicable sections of this report for review and comment. The FBI Finance Division Acting Assistant Director and General Services Acting Administrator provided written comments, which are reprinted in appendixes III and IV, respectively. FBI and GSA also provided technical comments, which we have incorporated as appropriate. We also discussed with Trilogy contractors any findings that related to them. We performed our work in accordance with generally accepted government auditing standards in Washington, D.C. and at two FBI field sites and various other GSA and contractor locations in Virginia from May 2004 through December 2005. [End of section] Appendix III: Comments from the Federal Bureau of Investigation: U.S. Department of Justice: Federal Bureau of Investigation: Washington, D.C. 20535-0001: February 2, 2006: Ms. Linda Calbom: U.S. Government Accountability Office: 441 G Street, NW: Washington, D.C. 20548: Re: GAO's Draft Report: Federal Bureau of Investigation: Weak Controls over Trilogy Project Led to Payment of Questionable Contractor Costs and Missing Assets: Dear Ms. Calbom: Thank you for the opportunity to review and comment on the Government Accountability Office (GAO) draft report entitled "Federal Bureau of Investigation: Weak Controls over Trilogy Project Led to Payment of Questionable Contractor Costs and Missing: Assets" (hereinafter "Report"). The Report has been reviewed by various components of the FBI, including the Finance Division, the Office of the Chief Information Officer, and the Office of General Counsel. This letter constitutes the formal FBI response to the Report, and I request that it be included in the GAO's final document. Based upon our review, we concur with the GAO's recommendations, and find them largely consistent with our internal assessment, prior to the GAO's conclusions, of the Trilogy program. While policies governing payments and property management were in place prior to Trilogy, the program suffered from a lack of comprehensive controls and weak or inconsistent enforcement of existing policies. The Report details some of the resulting failures. The Trilogy program as a whole has been the focus of extensive internal and external review. In response, the FBI has made and continues to make significant structural and procedural changes that address the GAO's recommendations, taking critical steps to strengthen internal controls. These changes - some planned, some already implemented - are intended to guard against similar missteps in future acquisitions. The FBI is committed to building a modern information technology infrastructure. After the events of 9/11, the FBI was required to move quickly to address a number of critical issues. Information technology was identified as a key shortcoming. Accordingly, the FBI and its partners responded with an accelerated deployment plan for FBI Field Offices. Execution started December 8, 2001, and was successfully completed within five months. During this phase, more than 120 contract employees in small teams replaced Local Area Networks with a high-speed IP-based system in each of the FBI's 56 Field Offices and deployed thousands of hardware items. Unquestionably, the urgency to replace the outdated, pre-9/11 infrastructure outstripped the pace of our internal controls. This is not to excuse the failings detailed in the Report. Specifically, weaknesses existed during Trilogy's asset acquisition and implementation processes. The Report cites weaknesses in internal controls governing review and approval of certain financial transactions, and insufficient controls surrounding receipt, recording, and reconciliation of assets purchased with Trilogy funding. In response, executive management at the FBI has directed a sustained effort to address and correct weaknesses demonstrated during the Trilogy program and identified in the Report and other reviews. Attention is being focused on improvements in four areas: increased audit capability; property management; contracting service; and management plans for information technology investments such as Sentinel. These improvements, both implemented and anticipated, are briefly discussed below. Audit Capability: In May 2005, while the GAO's audit of Trilogy was underway, the Finance Division established an Audit Unit reporting directly to the Chief Financial Officer, to evaluate internal controls and provide recommendations to improve operational efficiency and effectiveness. The unit continues to expand its responsibilities and is increasing review and oversight of major FBI programs and contracts, reporting findings to the Deputy Director. The Audit Unit is responsible for reviewing proposals, examining costs, and ensuring compliance with contract terms and conditions. Consistent with the GAO's recommendations, the FBI, in conjunction with the General Services Administration, had already planned a closeout audit for Trilogy. The audit will employ the Defense Contract Audit Agency (DCAA), once final cost is determined. As the contracting administrator for Trilogy, GSA will initiate the audit request, incorporating the issues and recommendations identified in the Report. Separately, the FBI annually institutes a Memorandum of Understanding with DCAA to conduct independent, third-party evaluations of FBI acquisitions. Property Management: To provide context for the Report's findings regarding property controls, the FBI notes that more than 44,000 pieces of accountable property were successfully deployed and tracked in the FBI's property management system during Trilogy. The Report initially identified 1,404 items - approximately 3 percent - missing or improperly documented. As of January 2006, the FBI had accounted for more than 1,000 of the items cited, and is continuing efforts to document the remaining Trilogy assets. The agreement with the Trilogy contractor resulted in modified property management procedures. In its discussion of control over Trilogy assets, the Report notes the FBI did not require compliance with its normal procedures for documentation of shipments from contractors. In discussions with GAO staff and in materials provided to GAO, the FBI explained that the normal policy was modified in order to maintain the contractor's control of the shipments until the contractor completed the installation process. In effect, while the FBI received the shipments, we did not accept delivery until the contractor processed the contents of those shipments. This modification for the Trilogy program should not be construed as a systemic lapse in the FBI's property management policies. The FBI is focused on improving property management, reinforcing existing policies and instituting stronger reporting and accountability across the FBI. KPMG, the independent auditor cited in the Report and contracted by the Department of Justice Inspector General to check the health and accuracy of FBI's financial statements, recently changed the FBI's property and equipment grade from a material weakness to a reportable condition, stating, "During fiscal year 2005, the FBI showed progress in resolving several of the issues noted in prior year audits, and has worked towards implementing effective and routine controls." Contracting Services: To tighten accountability in the execution of FBI contracts, all contracting officers will receive updated training outlining current policy, changes in regulations and new initiatives. In addition, a Finance Division reorganization created a new unit dedicated to coordinating acquisition planning, tracking, and reporting requirements on major programs. The unit will coordinate an acquisition plan that clearly defines and documents the roles and responsibilities of key personnel, such as the Contracting Officer, Contracting Officer's Technical Representative, Program Manager, Property Manager, and Financial Manager. The goal is to lay out clear lines of authority and accountability, and to avoid repeating the confusion described in the Report. Information Technology Investments: To ensure rigorous oversight of investments in information technology, the FBI consolidated oversight and technical expertise of information technology programs in a centralized Office of the Chief Information Officer (OCIO). The OCIO also has solicited information from vendors to provide an umbrella Independent Verification and Validation (IV&V) contract for use FBI- wide. This contract will provide a means for FBI managers to contract with a technical "watchdog" to independently review IT programs. In regard to Sentinel, a dedicated IV&V contractor reviewing the program will report directly to the CIO, providing oversight of both the contractor and the FBI's Sentinel Program Management Office (PMO). The Sentinel program office is structured to ensure compliance with existing laws, regulations and policies, and to avoid issues identified in the Report. The Program Manager has a clear line of reporting to the CIO, and a dedicated Business Management Unit is responsible for all contract oversight, cost estimation, cost control, invoice review and budget formulation and execution. In addition, a dedicated government Property Manager position reports to a separate Administration Unit. Coordinating with the Finance Division, these units will create Sentinel-specific policies and procedures for contract administration, invoice payment and property management. The FBI is committed to continuing improvement of our internal controls. We are documenting procedures where policies are lacking or unclear. Where policies exist, we are strengthening enforcement. We are working actively with Executive and Congressional oversight, as well as independent auditors, to identify and correct weaknesses in program management or reporting mechanisms. While safeguarding the nation is the FBI's first priority, we are also committed to meeting the fiduciary responsibilities of the public trust. To that end, thank you for the GAO's assistance, and for the opportunity to comment on your Report. Sincerely yours, Signed by: Richard L. Haley, II: Acting Assistant Director: Finance Division: Appendix IV: Comments from the General Services Administration: GSA Administrator: Ms. Linda Calbom: Director: Financial Management and Assurance: Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Ms. Calbom: Thank you for the opportunity to comment on the draft report, "Federal Bureau of Investigation: Weak Controls over Trilogy Project Led to Payment of Unallowable and Questionable Contractor Costs and Missing Assets." Enclosed please find the General Services Administration's response to the report. If you have any questions, please contact me. Staff inquiries can be directed to Lisa Akers at 703-306-7620. Sincerely, Signed by: David L. Bibb: Acting Administrator: Enclosure: GENERAL SERVICES ADMINISTRATION FEDERAL SYSTEMS INTEGRATION AND MANAGEMENT CENTER COMMENT TO GAO ON (DRAFT) REPORT ON THE FBI TRILOGY PROJECT FEBRUARY 2006: The Federal Systems Integration Management Center (FEDSIM) of the General Services Administration (GSA) is pleased with the opportunity to provide its comments in response to GAO's audit of the FBI Trilogy project. This response provides a description of the background that highlights the environment within which FEDSIM, the FBI, and the Contractors operated during Trilogy, responds to specific report findings, and addresses each of GAO's recommendations. GSA accepts GAO's recommendations and is pleased to describe some of the improvements to its internal controls and other business process changes already implemented. I. The Trilogy Program Environment: We believe it essential to view the Trilogy program within the context of urgency bracketing both the UAC and IPC/TNC projects. The impact of the 9-11 terrorist attacks on the business conducted between the parties involved in the executed Millennia Task Orders (T0001AJM026 and T0001AJM028) cannot be overstated. These attacks heightened the urgency for the systems while fundamentally influencing the priorities of the FBI. Within six months of award, and as a direct result of the 9-11 terrorist attacks, the original schedules for these two interdependent projects were compressed by 50% and there were fundamental requirements changes that created an incredibly dynamic environment. As milestones were accelerated and expediting activities became increasingly urgent, contractually required supporting efforts were often characterized to FEDSIM as patriotic endeavors, necessary to successfully fight the war on terrorism globally and within the homeland. Thus, the FBI required an accelerated delivery schedule in response to emerging and emergency requirements, often before such requirements were properly defined technically and before appropriate contract modifications could be executed. FEDSIM's contract administration activities required to incorporate the rapid and increasing number of changes were often viewed by FBI as too time-consuming, counterproductive, and unsupportive to the defense of the nation. Accordingly, FEDSIM support of the FBI Trilogy program after 9-11 was provided in a manner conducive to uninterrupted progress of the project, while concomitantly ensuring that FEDSIM maintained contract integrity to the greatest extent possible. While the general environment was dynamic, it is important to note that both task orders were negotiated from the outset to provide as clearly as possible the FBI's objectives in terms of milestones and deliverables at a reasonable estimated cost, and contained provisions by which contractor performance could be monitored and incentives or disincentives applied, as appropriate. II. Background on Contracting Approach: The GAO report mentions GSA's selection of contract type and other characteristics of the acquisitions. A Cost-Plus-Award-Fee (CPAF) contract type was selected for both orders. Use of this contract type was cited as a "contracting weakness" in a 2005 Department of Justice Office of Inspector General Report on Trilogy which GAO referenced in its report. It is important to note that FEDSIM was not afforded an opportunity to respond to the DOJ OIG report, and we take exception to the findings therein relative to contract management and administration. We maintain that the CPAF contract type was the only appropriate type of contract to use for the Trilogy program, considering the uniqueness of the requirements and uncertainty involved in performance of the Trilogy project In cases such as this, it is entirely appropriate for the Government to assume a greater degree of risk while providing appropriate incentives to contractors to achieve or exceed contract objectives. The FBI/FEDSIM team's responsibility was to assess the requirements and uncertainties involved in contract performance, and select the contract type and structure that placed appropriate degree of risk, responsibility, and incentives on the contractor for performance. Given the significant amount of changes to schedule and deliverable requirements that occurred, any other contract type would have most likely resulted in even greater overall cost to the Government. Additionally, under cost reimbursable task orders such as these, payments made against invoices or vouchers submitted during performance are interim payments in accordance with the clause at FAR 52.232-5. As such, are not considered final. Interim payments are fully recoverable by the Government (by means of direct payments from the contractor, credits on current invoices, or offsets against other contracts or task orders) in the event an audit determines that costs were paid that should not have been. The GAO report states that a cost reimbursable contract type (which includes CPAF) can only be used if appropriate Government surveillance exists during performance. As per FAR 16.301-3, this contract type may be used when the contractor's accounting system is adequate for determining costs applicable to the contract; and appropriate Government surveillance during performance will provide reasonable assurance that efficient methods and effective cost controls are used. FEDSIM believes it complied with these conditions. Both contractors have DCAA approved accounting systems, and there was significant Government surveillance of contractor performance by FBI, FEDSIM, and Mitretek during the entire period of performance of both orders. Performance evaluations were routinely conducted by FBI, FEDSIM, and Mitretek personnel as part of Award Fee Evaluation Board procedures, and performance was documented in Award Fee Determination reports. Additionally, during performance there were several contract administration and program management changes implemented on both task orders. In an effort to improve oversight, performance was tied more closely to outcomes, and to incentivize contractors to achieve contract objectives, FEDSIM implemented the following changes to the TNC/IPC and VCF task orders: * A Governance Board structure was implemented, and was comprised of DOJ, FBI, GSA, the prime contractors, and OMB in November 2003 to keep the executive levels of all organizations informed of progress, impediments, risks, and proposed corrective actions. * A cost sharing provision was negotiated and implemented by task order modification PS25 on the TNC/IPC task order that stipulated that if Full Site Capability (FSC, as defined in the task order) was not achieved by 30 April 04, the contractor would bear 50% of any cost overrun associated with the delay. Additionally if FSC was not achieved by the milestone date, the contractor would give up 100% of potential award fee. * On the VCF task order, specific milestones with dates and objectives written/incorporated by modification PS28 into Sections C (Statement of Work) and F (Deliverables); Section B (Schedule of Supplies/Services) contract line item numbers (CLINs) were divided into sub-CLINs to reflect level of effort and cost associated with each milestone. * On the VCF task order, the contractor gave up $7.6 million in potential award fees to cover costs of the VCF Initial Operating Capability (IOC) in modification PS29. This portion of the VCF effort was pure cost reimbursable (no fees) and included an initial 33% cost sharing provision. * On the VCF IOC, FEDSIM negotiated revised terms in modification PS29 that tied the release of contract funding to achievement of milestones/control gates - if the contractor didn't meet the control gate/milestone requirements, then it would have to cover its costs until control gate requirements were met, at which point government funding would be provided. Achievement of control gate requirements was certified in writing by the FBI COTR. III. FEDSIM Response to GAO Comments: 1.a. GAO Audit Comment: Insufficient invoice review and approval process increased FBI's vulnerability to payment of questionable contractor costs. 1.b. FEDSIM Response: The GAO report identifies inadequacies in the invoice review and approval process and states that roles between the FBI and FEDSIM were not clearly defined. Invoice review and approval is one of many project- related tasks that FEDSIM provides as part of post-award project management support. In addition to monitoring performance to ensure reasonable progress toward contract objectives, the members of the FBI, FEDSIM, and Mitretek team each played a unique and mutually understood role in reviewing and processing invoices for payment under Trilogy. All invoices were processed in accordance with the Prompt Payment Act, task order provisions specific to invoices, and GSA agency-approved procedures, and always in conjunction with FBI approval. While we believe that this approach was generally sound, we also believe that the procedures could have been better documented and more formally implemented. FEDSIM provided contractual approval for payments, and it relied upon the FBI to make technical and programmatic review and approval decisions, prior to authorizing or approving invoices for any payment. The FBI made technical and programmatic decisions relative to inspection and acceptance procedures, along with invoice review and approval, using the SETA contractor, Mitretek. The role of Mitretek in invoice review and approval was significant, and is substantiated by Mitretek Monthly Progress and Financial Reports submitted under its contract with Department of the Interior (DOI). [NOTE 1] The activities performed by Mitretek in support of Trilogy include "reconciling Trilogy TNC/IPC Bills of Materials with contract modifications to update funding and Consent to Purchase Line Items" and "tracking and analyzing equipment purchase requests, labor invoices, and travel authorizations" to "allow for expeditious review and management of equipment purchase orders, labor invoices, and travel authorizations." [NOTE 2] Further, Mitretek status reports reflect invoice review activities to substantiate the "accuracy of DynCorp's invoices and validation of the balance of materials due." Therefore, FEDSIM maintains that it was clearly reasonable to have relied on the input received from the FBI, via Mitretek, in approving invoices for payment. Once FEDSIM conducted its own analysis and received signed approval from the FBI, the invoice was approved and forwarded for payment. GSA procedure requires the COR to approve or reject an invoice within 5 days of receipt so that payment can be made within the 30 days required by the Prompt Payment Act. GAO documented an example of one individual working an average of 70 hours per week for a four-week span; however, FEDSIM believes this is not excessive given the enormity of the schedule acceleration for Trilogy. To the contrary, given the required schedule acceleration, significant amounts of overtime were normal. The review team primarily focused on the general level of effort, as measured against both expectations and performance. In addition to the significant level of effort required to achieve contract objectives, invoices often reflected changes in a particular employee's hours charged as a result of timecard corrections or other administrative issues. These items were in fact clarified with the contractors verbally or via email. NOTES: [1] The relevant documentation, including copies of the Mitretek contract, status reports, and invoices, was provided to GAO during its audit. [2] The language in quotations is cited verbatim in Mitretek status reports of activities performed. While certain charges under the task orders may appear to be questionable in nature, it is FEDSIM's intent to ensure all costs claimed are reasonable, allowable, and allocable via the DCAA final closeout audits. In our role as contracting authority, we will request and rely on the following types of audits: incurred cost audits (in particular direct labor and subcontractor direct labor costs), audits of indirect rates, and audits of travel and ODC costs. The results of the audits will ensure that FEDSIM has factual basis for determining if any costs are unallowable, unreasonable, or unallocable. Following such audits, the Contracting Officer will use the information as the basis to pursue remedies to recoup funds and assess penalties as may be applicable. 2.a. GAO Audit Comment: Invoices were approved for payment without validation that goods and services were received. Invoices did not provide adequate support for all charges, specifically: * Labor (p.15); * Subcontractor Labor (p. 16); * Travel (p. 17); * Other Direct Costs (ODC) (p. 18); * Equipment (p. 19). 3.2.b FEDSIM Response: Pursuant to FAR 31.201-2(d), contractors are required to properly account for costs incurred on contracts and to maintain records, including supporting documentation, adequate to demonstrate that costs claimed have been incurred, are reasonable, are allowable, and are allocable to the contract, including costs related to subcontractors. (1) Labor and Subcontractor Labor: With regard to labor, GAO's report states that GSA could not explain or provide evidence of how it "resolved clearly questionable labor charges, including hours billed far in excess of a normal pay period." FEDSIM examined invoices received to ensure that they were proper invoices, as defined in the task orders, and that they contained all information required by the task order. If an invoice did not contain the required applicable information, it was rejected. FEDSIM reviewed invoices containing anomalies in the number of labor hours being billed by individuals, compared the billable rate against the Millennia ceiling rates, and ensured there was sufficient funding on the task order to pay the charges identified on the invoice. In some cases where an invoice showed an unusually high number of labor hours for an individual charging to the project, verbal or email clarification was sought and obtained. In most cases, the anomaly was a result of timecard corrections or other administrative issues. In others, additional documentation was requested to validate the number of hours charged. GAO draws a similar conclusion about the validity of subcontractor labor charges. All supporting documentation relative to subcontractor invoices is maintained by the subcontractors and is available for Contracting Officer and/or auditor review at any time. The significant presence of both FBI and Mitretek personnel on-site at contractor facilities, as well as the efforts of the FEDSIM COR, provided reasonable assurance that work was progressing and was adequately monitored in accordance with the terms and conditions of the task orders. The FBI did not provide FEDSIM with its signature for invoice approval until it had received concurrence from Mitretek, since it relied upon Mitretek technical expertise relative to work being performed compared to labor hours and categories charged. (2) Travel: The GAO report suggests that reliance on the review team to examine travel vouchers was insufficient. GSA disagrees with this conclusion. Contractor requests for travel were made to both FEDSIM and the FBI. The CSC travel policies governing travel on the Trilogy Program provided that all travel arrangements had to be made by the Designated Travel Administrator, which was at the time American Express Travel Services. No individual employees of the contractor (or subcontractors) booked their own travel arrangements. The Travel Administrator was obligated to obtain the best fare available at the time of booking, to make maximum use of Advance Purchase fares when possible, and to maximize use of preexisting agreements with various airlines providing discounts off of published fares. FEDSIM was not responsible for identifying or categorizing FBI priorities related to travel. While FEDSIM provided contractual approval (i.e., authorized the contractor to incur travel costs), the FBI provided the technical and programmatic approval decisions, determination of travel requirements and authorization to conduct travel accordingly. The approval process was logical and appropriate in that it implemented a procedure where the FBI reviewed and approved/rejected travels requests and sent an email to FEDSIM regarding its decision. FEDSIM retained documentation of approved travel requests. It must also be recognized that GAO's conclusion does not account for the ever-changing travel schedules and itineraries necessitated by changes in FBI requirements. The relevant FAR Cost Principle regarding contractor travel is FAR 31.205-46, which states that "airfare costs in excess of the lowest customary standard, coach, or equivalent airfare offered during normal business hours are unallowable except when such accommodations require circuitous routing, require travel during unreasonable hours, excessively prolong travel, result in increased cost that would offset transportation savings, are not reasonably adequate for the physical or medical needs of the traveler, or are not reasonably available to meet mission requirements." Clearly if such exceptions cannot be substantiated during closeout audits, FEDSIM will recoup any costs determined to be unallowable by such audits. The dynamic nature of airfares requires that any determination of reasonableness and allowability be based on the circumstances existing at the time of travel. Fares vary significantly from day to day and airport to airport. Furthermore, a hypothetical "standard coach class ticket" does not provide a benchmark against which to make a valid price comparison, and even adjustment for inflation is not adequate to develop a benchmark fare. During the period of performance of the task orders, the airline travel industry underwent significant changes with respect to pricing of airline tickets. The availability of competitive pricing due to increase in the use of the Internet to research and book fares has resulted in dramatic decreases in the cost of airline tickets today. Further, airlines offer only a limited number of seats at the lowest (and most restrictive, i.e. Saturday night stayover, non- refundable, non-changeable) economy fares and these seats are generally sold well in advance of the actual travel date. Since the majority of the trips identified in the GAO report were arranged within 3 days of travel, it is unlikely that deeply discounted fares were still available. Therefore, we believe it is impossible at this date to look back over five years of travel and estimate what may have been a "reasonable" airfare price for any one particular travel segment. (3) Other Direct Costs (ODC) and Equipment: GAO states that FEDSIM did not perform a review to validate the amounts invoiced for ODC charges and that FEDSIM did not verify equipment had actually been received before making payment. FEDSIM verified that Consents to Purchase (CTP) had been approved and there was sufficient funding on the respective CLIN to pay the invoice. FEDSIM also compared the charges on the invoice to the estimated costs on the appropriate bill of materials. If the billed costs were substantially more than the estimates provided, FEDSIM would ask for clarification from the contractor. FEDSIM relied on the FBI to verify that the items had been received, inspected and accepted, since all deliveries were made directly to the FBI under Trilogy. Per the task orders, all deliverables were to be sent to FBI-defined locations, and an authorized FBI representative provided acceptance. FEDSIM did not receive actual deliverables, but instead received cover letter notices when delivery was made. 3.a. GAO Audit Comment: GAO concluded that SAIC incorrectly billed overtime charges based upon an informal agreement between SAIC and the FBI. 3.b. FEDSIM Response: While the policy of Extended Work Week (EWW) was provided to the Government in its proposals, SAIC did not follow its own written procedure for implementing the EWW when it became necessary. The informal policy agreement between the FBI and SAIC was not provided to the contracting officer. It should also be noted that the text of the original EWW governing language referred to "..extended hours for short period of time .." Thus, the original language envisioned some very brief periods requiring a limited amount of overtime which were not originally estimated. 4.a GAO Audit Comment: GAO states that questionable labor rates charged by DynIS may have exceeded rates that GSA asserts were established ceiling rates pursuant to the Task Order. 4.b FEDSIM Response: FEDSIM concurs with GAO's finding regarding their conclusion that DynIS charged labor rates that exceed ceiling rates established in DynCorp's Trilogy contract. The Price Negotiation Memorandum (PNM) indicated that FEDSIM had planned to conduct appropriate negotiations, subsequent to conduct of a DCAA audit. The DynIS rate audit was not requested and the subsequent negotiations were not conducted. The DCAA audit will provide for a retroactive rate re-determination and costs incurred will be adjusted accordingly. IV. Specific GAO Recommendations and FEDSIM Response: GAO accepts each of GAO's recommendations. As part of its continuous process improvement, FEDSIM is conducting on-going assessments of processes and procedures; these assessments, whether required by external or internal sources, are essential to ensuring that FEDSIM provides the acquisition, project management and contract administration services to ensure best value to the Government and the taxpayer. 1.a GAO Recommendation: Reassess FEDSIM procedures regarding clearly defining the roles and responsibilities of each party in Interagency Agreements, particularly those related to reviewing and approving invoices. 1.b FEDSIM Response: The standard FEDSIM Interagency Agreement has been modified to more clearly identify the FEDSIM and client roles and responsibilities. Relative to invoicing, the FEDSIM responsibilities include "perform final acceptance of supplies/services, approve invoices for supplies/services that are reimbursed through one of GSA's revolving funds and bill the client." Relative to invoicing, the Client responsibilities include "receive, inspect, and reject or technically accept supplies/services in a timely manner and execute all responsibilities in a timely fashion so that all provisions of the Prompt Payment Act can be met." The client responsibilities also include that the client will "not authorize work, change any contractual terms, authorize accrual of cost or otherwise provide direction to vendors outside authority delegated by GSA's Contracting Officer." Additionally, Governance Board structures are now implemented on major task orders; the Boards are comprised of FEDSIM management representatives, client agency personnel, and contractor representatives to keep executive levels of all organizations informed of progress, any impediments or performance/financial problems, and program risks. FEDSIM also provides Award Fee Board training on all task orders that incorporate Award Fee as an incentive, with the exception of the Trilogy task orders. The purpose of the training is to ensure the board members: understand Award Fee basics and have read and understand the Award Fee Plan; understand the Governments' rights and obligations with respect to Award Fee; understand their role in conducting surveillance of the contractor performance; how to evaluate the contractor's performance in accordance with the current Award Fee Plan. FBI personnel on the task order did not complete this training, as the FBI did not believe it would find value in the training. FEDSIM may modify its IA documents to reflect that it will no longer support acquisitions that incorporate Award Fee incentives for clients that refuse to complete the Award Fee training requirements. 2.a GAO Recommendation: Reassess FEDSIM procedures regarding the adequacy of its invoice review and approval policies, including specific steps to be performed by each party so that (1) invoices provide the information required in the contract to support the charges, (2) goods and services billed on invoices have been received, (3) amounts are appropriate and in accordance with contract terms, and (4) the resolution of any questionable or unsupported charges on contractor invoices identified during the review process is clearly documented. 2.b FEDSIM Response: In November 2003, a training session was conducted that focused on invoice processing procedures. In December 2005, mandatory training was conducted addressing in-depth invoice review procedures. This training focused on the ceiling rates, subcontractor rates, ODC review, travel review, and the actions to take when questionable charges arise. FEDSIM plans to continually review and update the training and ensure we fully incorporate GAO's recommendations, as appropriate. We also plan to present this training on an annual basis to our associates. FEDSIM has formalized a specialized Project Analyst position to provide consistent tracking and reporting of financial and schedule information for large projects, to include tracking funded and ceiling values, reviewing consents to purchase, performing detailed invoice review, monitoring schedule, and providing analysis of overall earned value management indicators. These activities are primarily performed by the assigned COR for the order. However, specializing the role provides greater analysis and also improves our reporting to clients. This role will improve project oversight, help mitigate programmatic risks, and provides greater value to our clients. 3.a GAO Recommendation: FEDSIM should clearly document labor rates, ceiling limits, treatment of overtime hours, and other key terms for cost determination for all contracts, task orders, and related agreements. 3.b FEDSIM Response: FEDSIM will review its processes to ensure that details such as those identified by GAO are captured contractually and understood and assessed by our Project Management and Contracting staff. In addition, we will review how an attachment to the task order could provide more clarity in the documenting of the ceiling rates negotiated at the execution of future task orders. FEDSIM has implemented a Project Review Tool (PRT) to assess various components of a projects' health. The PRT captures the status and actions on a projects' financial, schedule, funding/spending, risk, compliance, acquisition activity, relationship management, and action plans required. This tool serves as a management control tool to ensure a structured approach to proactive and consistent reviews in key areas, serving to ensure issues are properly elevated to senior management levels when necessary. 4.a GAO Recommendation: FEDSIM should clearly reflect in future contracts the appropriate FAR travel cost requirements, including the purchase of the lowest standard, coach, or equivalent airfare. 4.b FEDSIM Response: FEDSIM believes that the requirement as outlined at FAR 31.205-46 and as stipulated in the task orders is more than adequate, but will review these requirements at each task order kick-off meeting with all personnel, to include client agency and contractor personnel. This topic will also be included in the Quality Assurance Surveillance Plan (QASP). The existing task order clauses provide appropriate guidance with respect to contractor travel under the task order, to include procedures for travel requests, approvals, and travel invoice requirements. We note that there may be extenuating circumstances that support exceptions to FAR 31.205-46, and such exceptions will be documented in the task order file. 5.a GAO Recommendation: FEDSIM should confirm that contractors properly review and support submitted subcontractor charges. 5.b FEDSIM Response: FEDSIM concurs with GAO's recommendation. We believe that obtaining DCAA purchasing and accounting system review results will assist in determining the extent to which subcontracting shall be monitored during task order performance. We also intend to conduct periodic desk audits to ensure Prime Contractor's have the required and accurate documentation to support subcontractor charges. V. Specific GAO Joint Recommendations and FEDSIM Response: 1.a GAO Recommendation: FEDSIM and FBI should confirm SAIC's informal EWW policy and work with SAIC to determine and resolve any overpaid amounts. l.b FEDSIM Response: FEDSIM concurs with GAO's recommendation. 2.a GAO Recommendation: FEDSIM and FBI investigate whether DynIS labor rates exceeded ceiling rates and pursue recovery of any amounts determined to have been overpaid. 2.b FEDSIM Response: FEDSIM concurs with GAO's recommendation. 3.a. GAO Recommendation: FEDSIM and FBI should determine whether contractor costs identified as questionable in this report should be reimbursed to FBI by contractors. 3.b FEDSIM Response: FEDSIM concurs with GAO's recommendation. As the contracting authority, FEDSIM will work with DCAA to obtain necessary audits and will also provide DCAA GAO's findings in order to facilitate review of audit areas. DCAA and FEDSIM have established agreements for DCAA to perform closeout audits on Department of Defense task orders at no cost and on civilian agency task for a reasonable cost. FEDSIM is in discussions with our GSA OIG and DCAA to obtain dedicated resources to periodically perform audits of contractor cost and subcontractor documentation throughout the life of the cost reimbursable task orders. 5.4.a. GAO Recommendation: FEDSIM and FBI should consider engaging an independent third party to conduct follow-up audit work on contractor billings, particularly areas of vulnerability in this report. 5.4.b FEDSIM Response: FEDSIM concurs with GAO's recommendation. GAO Comments: 1. We referred to the Department of Justice Office of Inspector General report only to provide background information related to previously reported issues with the Trilogy project. 2. See "Agency Comments and Our Evaluation" section. 3. Processing invoices timely as envisioned by the Prompt Payment Act does not lessen the government's responsibility to verify costs billed by contractors. It is conceivable that the essential validation work could have been performed immediately after payment and any adjustments to correct prior billing errors could have been made to future invoices. 4. No documentation of any such inquiries was provided to support the General Services Administration's (GSA) comment. Documenting such inquiries allows a subsequent reviewer to draw similar conclusions and would be beneficial to any subsequent audit, including by the Defense Contract Audit Agency (DCAA). 5. Contrary to GSA's comment, the review team--Federal Bureau of Investigation (FBI), GSA, and Mitretek--approved CSC's invoices that lacked information required by its task order, including employee billing rates and detail for subcontractor labor. We were not provided documentation indicating that any Computer Sciences Corporation (CSC) invoice had been rejected. 6. While the review team compared billed labor rates against Millennia ceiling rates for certain labor costs, it did not evaluate labor rates compared to ceiling rates for subcontractor labor, which represented about $163 million of Trilogy costs. Had the review team reviewed labor charges more thoroughly, it may have identified the potential overcharging of labor rates discussed in this report related to DynCorp Information Systems (DynIS). 7. According to the Federal Acquisition Regulation (FAR), it is the contractor's responsibility to maintain supporting documentation for costs billed, including subcontractor labor costs. 8. Based on our discussion with on-site members of the review team, CSC travel vouchers were not obtained to review amounts billed on travel invoices. Had the vouchers been reviewed, the review team would have had a basis for questioning the first-class and excessive airfare costs we identified. 9. The travel administrator's obligation to obtain the best fare does not relieve the government of its responsibility to review travel costs. In addition, we noted instances where the itinerary from the travel administrator indicated that a full-fare ticket was obtained at the traveler's request, even though the ticket cost more than twice as much as the lowest logical fare that was also noted on the itinerary. 10. The approval process discussed by GSA relates to the travel authorization, which was the request to travel. We found that the review team lacked an adequate process to review travel vouchers that include the traveler's receipts to confirm that the authorized trips were taken and that the costs were in accordance with applicable travel regulations. Also see comments 8 and 9. 11. The next sentence of the relevant section of the FAR cited by GSA states, "However, in order for airfare costs in excess of the above standard airfare to be allowable, the applicable condition(s) set forth above must be documented and justified." No such documentation was provided to us for any of the first class or other excessive airfares we identified. 12. Our report stated that other direct costs (ODC) were paid without validation of the actual amounts included in the invoices and that the review team relied on the contractors to obtain purchase orders for ODC charges. It further stated that neither GSA, FBI, nor Mitretek performed procedures to ensure that equipment billed by the contractors was actually received before payment. 13. CSC ODC invoices lacked sufficient detail to validate amounts billed compared to what was approved and we were not provided documentation indicating that such information was requested by the review team. Further, the CSC invoices did not include the detail necessary for the review team to specifically identify the items purchased. We also found that some assets were paid for before they were received and that the FBI did not perform an overall reconciliation of total assets ordered and paid for to those received. 14. A GSA contracting officer representative told us that he was aware of the informal extended work week policy agreement, but could not provide documentation of the policy. 15. Our report stated that DynIS charged labor rates that may have exceeded rates that GSA asserts were established ceiling rates pursuant to the task order. 16. Based on GSA's acceptance of our recommendations on page 1 of its comments, we assume that the intent was to state that "GSA accepts each of GAO's recommendations." [End of section] Appendix V: GAO Contacts and Staff Acknowledgments: GAO Contacts: Linda Calbom, (202) 512-9508, or [Hyperlink, calboml@gao.gov]. Acknowledgments: Staff members who made key contributions to this report include Steven Haughton (Assistant Director), Marie Ahearn, Brooks Bare, Ed Brown, Marcia Carlsen, Richard Cambosos, Lisa Crye, Tyshawn Davis, Bonnie Derby, Abe Dymond, Lori Ryza, Kara Scott, Brooke Whittaker, and Matt Wood. (190140): FOOTNOTES [1] See appendix I for a timeline of significant milestones related to the Trilogy project. [2] Department of Justice, Office of the Inspector General, The Federal Bureau of Investigation's Management of the Trilogy Information Technology Modernization Project, Report No. 05-07 (Washington, D.C.: February 2005). [3] GAO, Information Technology: FBI Is Building Management Capabilities Essential to Successful Systems Deployments, but Challenges Remain, GAO-05-1014T (Washington, D.C.: Sept. 14, 2005). [4] For the purpose of this report, unallowable costs are contractor costs that are not allowed under a term or condition of the contract or pursuant to applicable regulations. [5] GAO, Internal Control: Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). GAO, Guide for Evaluating and Testing Controls Over Sensitive Payments, GAO/AFMD-8.1.2 (Washington, D.C.: May 1993). GAO, Strategies to Manage Improper Payments: Learning From Public and Private Sector Organizations, GAO-02-69G (Washington, D.C.: October 2001). [6] 48 C.F.R. chp.1. [7] 41 C.F.R. subtitle F. [8] Department of Defense Civilian Personnel Joint Travel Regulations. [9] Questionable costs include payments of amounts that we determined to be potentially unallowable; lack the support necessary to determine whether they are allowable under applicable laws, regulations, and the terms and conditions of the contract; or for which there is a disagreement between the parties as to whether the payment is allowable under applicable laws, regulations, and the terms and conditions of the contract. [10] The Millennia Governmentwide Acquisition Contract provides for large system integration and development projects through task or delivery orders awarded to Millennia contractors, including Computer Sciences Corporation and Science Applications International Corporation. [11] In March 2003, DynCorp was acquired by CSC. [12] For purposes of this report, the task orders awarded under the Millennia contract will be referred to as "contracts." [13] Award fees consist of money that is added to a contract and that a contractor may earn in whole or in part during performance and that is sufficient to provide motivation for excellence in the areas such as quality, schedule, technical performance, and cost management. [14] In July 2002, the FBI/GSA FEDSIM reimbursement agreement related to support for the Mitretek contract ended and the FBI entered into a similar agreement with DOI to support the Mitretek SETA contract. [15] A project integrator provides the overall planning and coordination during the implementation of a new system. The tasks an integrator performs include the defining of requirements for system implementation, scheduling, and ensuring that testing is performed. [16] DOJ initially required the FBI to perform the project integration function; however, the FBI did not have sufficient project integration expertise. The FBI made a $20 million reprogramming request and SAIC was brought on as integrator in October 2003. [17] Department of Justice Office of the Inspector General, The Federal Bureau of Investigation's Control Over Weapons and Laptop Computers, Report No. 02-27 (Washington, D.C.: August 2002). [18] Department of Justice Office of the Inspector General, Controls over Accountable Property at the Baltimore Field Division of the Federal Bureau of Investigation, Report No. 04-37 (Washington, D.C.: September 2004). [19] GAO, Internal Control: Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). See also, GAO, Policy and Procedure Manual For Guidance of Federal Agencies, Title 7, Fiscal Guidance, chps. 6&7 (Washington, D.C.: May 1993). [20] Office of Management and Budget, Circular A-123, Management's Responsibility for Internal Control, defines internal control guidance for federal agencies. [21] About $41 million of this amount represents labor charged by a subsidiary company, which was treated as a subcontractor. [22] GAO, High Risk Series: An Update, GAO-05-207 (Washington, D.C.: January 2005). [23] Prior to purchasing equipment during the Trilogy project, CSC and SAIC would submit a request to purchase called a Bill of Material (BOM) for CSC and a consent to purchase for SAIC to the FBI for approval. The BOM listed the descriptions of the equipment to be purchased, the quantity, and the price per item. [24] Although Mitretek's invoices did not include this information, during our review of selected charges we were able to obtain additional information from Mitretek to verify the labor rates charged and to recalculate the labor costs. [25] GSA provided us a list of 16 SAIC invoices that were rejected during the Trilogy project for various reasons, such as the review team needed further detail and clarification for invoiced charges and issues related to labor charges. GSA did not provide any documentation that additional support was obtained or that these issues were resolved. [26] The 19 first-class airfares include trips with at least one leg of first-class travel and exclude any first-class tickets where the itinerary identifies the fare as a "free first class upgrade." [27] We were not able to estimate the cost of coach-class fares for some of the first-class trips because of unusual routing of certain one- way trips. Table 2 provides examples of the fare differences we were able to determine. [28] First-class travel may be allowed under certain circumstances, such as when lower class accommodations are not reasonably available or for medical reasons. [29] The FAR states that airfare costs in excess of the lowest customary standard, coach, or equivalent airfare offered during normal business hours are unallowable except when such accommodations require circuitous routing, require travel during unreasonable hours, excessively prolong travel, result in increased cost that would offset transportation savings, are not reasonably adequate for the physical or medical needs of the traveler, or are not reasonably available to meet mission requirements. However, in order for airfare costs in excess of the above standard airfare to be allowable, the applicable condition(s) must be documented and justified. [30] Both tickets purchased by Mitretek were restricted tickets. [31] GSA officials also indicated that they were aware of this informal agreement. [32] SAIC officials indicated that in June 2003 a waiver of the 10 hours of uncompensated time associated with the EWW policy was implemented for select teams. However, SAIC could not provide us information on which teams, tasks, or employees the waiver applied to or the length of time the waiver covered. Therefore, we were not able to consider this waiver in our analysis. [33] This estimate of the incorrect EWW hours charged is based on SAIC's summary of labor hours charged. The actual incorrect hours could be affected by time sheet corrections or other factors. [34] This estimate was calculated based on average fully burdened labor rates using cumulative hours and costs billed for employees that appeared to incorrectly charge EWW hours and the amount of hours that appeared to be incorrectly billed to FBI. The actual amount of the overpayment would be influenced by the actual labor rates of employees working EWW, as well as SAIC indirect billing rates that are charged on labor costs for overhead, fringe benefits, and general and administrative costs. [35] The Defense Contract Audit Agency, or DCAA, is responsible for performing all contract audits for the Department of Defense. They also provide contract audit services to other government agencies when hired to do so. [36] GSA officials said they believed that since proposed labor category rates for DynIS varied in each revised proposal, this supports their assertion that the rates were negotiated. [37] CSC contends that the labor category hourly rates presented in DynCorp's proposals merely represented a detailed cost breakdown of DynIS' estimated costs. To further support their contention, CSC referred us to DynCorp's labor invoices, which consistently listed DynIS ceiling rates as "TBD" (to be determined). [38] The modification contained actual labor rates for year 1 of the contract and then presented projected labor rates for years 2 and 3 of the contract. We could not verify CSC's explanation for how they calculated the year 1 totals. [39] We estimated the potential overcharge based on the total hours charged and the difference between the possible ceiling rates and the actual average labor rates charged by DynIS on an annual basis. For work performed in 2001, because the proposal had different rates for work performed in the field or at headquarters and CSC's labor invoices did not indicate where employees worked, we calculated average ceiling rates based on an assumption that 50 percent of employees worked in the field and 50 percent worked at headquarters. For 2002 and 2003, we used rates from the previously discussed modification that included the same rates for the field or headquarters. [40] Our Standards for Internal Control in the Federal Government indicates that the proper execution of transactions should include determining that only valid transactions are authorized. Further, all transactions must be properly documented and documentation must be readily available for review. [41] CACI provided us a "training log" for the courses coordinated by the event planner. This training log was a spreadsheet that summarized training courses scheduled in 23 cities and included the dates of the courses, and the number of attendees. We requested but did not receive a copy of FBI's training logs to verify its participation at this training. [42] In May 2003, CACI entered into an agreement with the event planner to terminate the contract after training was conducted at only 23 of the 72 sites. CACI eventually paid the full amount of the purchase order (about $2.993 million) plus an additional net amount of $5,776 (a settlement amount of $62,214 for less than full performance reduced by $56,438 for unsupported event planner prepayments to reserve training facilities). Subsequently, CACI entered into a contract with a second event planner to procure facilities for FBI training. [43] Represents Trilogy equipment purchased by CSC, SAIC, and directly by FBI that was delivered to CSC for the IPC/TNC portion of the project. [44] We limited this comparison to CSC-purchased assets, which represented about 52 percent of Trilogy assets. We could not perform this test for assets purchased directly by FBI because it did not track these assets by bar code and therefore did not have the data necessary for this analysis. [45] The CSC-prepared listing of equipment included equipment purchased directly by CSC, as well as all equipment purchased directly by FBI that was delivered to CSC or its subcontractors. [46] The use of bar codes involves affixing a machine-readable bar code to a controlled item, which can then be scanned and compared to an equipment inventory listing as part of a periodic physical inventory. [47] FBI subsequently revised its instructions to contractors; however, the contractors never removed the affixed bar codes from equipment items that had already been erroneously tagged. [48] FBI's records showed that nonaccountable assets purchased by CSC totaled about $37.4 million or approximately 32 percent of the dollar value of the CSC-purchased assets and 22 percent of the reported total hardware purchased on the Trilogy project. [49] As part of its spring 2005 inventory, FBI identified over 310 accountable-Trilogy assets valued at about $1.2 million that had not been recorded in PMA, over 61 percent of which had been installed prior to FBI's fiscal year 2003 inventory, including several assets that were installed in 2001. [50] The selected items were desktop computers, Dell PowerEdge 2550 servers, Cisco 4006 switches, and Cisco 6509 switches. We chose these items because they represented major items purchased for the Trilogy project. [51] This guidance included the final corrected description provided to contractors of accountable units to be affixed with bar codes. [52] As of January 2006, we have physically observed nine taclanes, of which five were not being used by the FBI offices in Baltimore, Md. and Chantilly, Va. The FBI staff person responsible for monitoring the taclanes could not explain why five of the taclanes were not being used at the FBI sites. [53] FBI input the last of these missing items into PMA on December 8, 2005. [54] We identified these as Trilogy assets by comparing the FBI bar codes for CSC-and SAIC-purchased Trilogy assets against its complete listing of lost and stolen assets for the 3 years, which totaled 2,331 assets valued at $6.7 million. We could not perform this test for assets purchased directly by FBI because vendor invoices did not include serial or bar code numbers and therefore did not have the data necessary for this analysis. [55] In two instances, we searched for restricted tickets, because the original tickets billed by Mitretek were nonrefundable. [56] For purposes of our payment testing, CSC labor costs included DynIS labor. [57] Because CSC was slow in providing the transaction-level detail for the labor invoices, we recalculated invoice amounts for 7 of the 11 invoices. Because we found no significant issues, we did not test the remaining 4 invoices. [58] For purposes of our payment testing, DynIS labor costs were not included in our review of subcontractor labor costs. [59] Mitretek's ODC included consultant costs. GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.