DCAA Audits

Widespread Problems with Audit Quality Require Significant Reform Gao ID: GAO-09-468 September 23, 2009

The Defense Contract Audit Agency (DCAA) under the Department of Defense (DOD) Comptroller plays a critical role in contractor oversight by providing auditing, accounting, and financial advisory services in connection with DOD and other federal agency contracts and subcontracts. Last year, GAO found numerous problems with DCAA audit quality at three locations in California, including the failure to meet professional auditing standards. This report addresses audit quality issues at DCAA offices nationwide. GAO was asked to (1) conduct a broad assessment of DCAA's management environment and audit quality assurance structure, (2) evaluate DCAA actions to date to correct previously identified problems, and (3) identify potential legislative and other actions for improving DCAA effectiveness and independence. To achieve these objectives, GAO analyzed DCAA's mission, strategic plan, audit policies, and quality assurance program; conducted interviews; reviewed selected audits at DCAA offices; and analyzed legislative and other actions.

GAO found audit quality problems at DCAA offices nationwide, including compromise of auditor independence, insufficient audit testing, and inadequate planning and supervision. GAO's conclusions stem from a review of 69 audit assignments supporting contract award and administrative decisions; an assessment of DCAA's audit quality assurance structure, which found similar audit quality problems but gave satisfactory ratings to deficient audits; and DCAA's rescission of 80 problem audit reports. The rescinded audits supported decisions on pricing and contract awards and impacted the planning and reliability of hundreds of other DCAA audits, representing billions of dollars in DOD expenditures. GAO found DCAA's management environment and quality assurance structure were based on a production-oriented mission that put DCAA in the role of facilitating DOD contracting without also protecting the public interest. DCAA has taken several positive steps. However, DOD and DCAA have not yet addressed fundamental weaknesses in DCAA's mission, strategic plan, metrics, audit approach, and human capital practices that had a detrimental effect on audit quality. To improve DCAA oversight, the DOD Comptroller requested Defense Business Board and "tiger team" reviews and established a DCAA Oversight Committee. In addition, in the short-term, Congress could provide DCAA with certain legislative protections and authorities similar to those available to IGs. In the longer term, Congress may wish to consider organizational changes to elevate DCAA to a component agency reporting to the Deputy Secretary or to establish an independent governmentwide contract audit agency.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-09-468, DCAA Audits: Widespread Problems with Audit Quality Require Significant Reform This is the accessible text file for GAO report number GAO-09-468 entitled 'DCAA Audits: Widespread Problems with Audit Quality Require Significant Reform' which was released on September 23, 2009. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Committee on Homeland Security and Governmental Affairs, U.S. Senate: United States Government Accountability Office: GAO: September 2009: DCAA Audits: Widespread Problems with Audit Quality Require Significant Reform: GAO-09-468: GAO Highlights: Highlights of GAO-09-468, a report to the Committee on Homeland Security and Governmental Affairs, U.S. Senate. Why GAO Did This Study: The Defense Contract Audit Agency (DCAA) under the Department of Defense (DOD) Comptroller plays a critical role in contractor oversight by providing auditing, accounting, and financial advisory services in connection with DOD and other federal agency contracts and subcontracts. Last year, GAO found numerous problems with DCAA audit quality at three locations in California, including the failure to meet professional auditing standards. This report addresses audit quality issues at DCAA offices nationwide. GAO was asked to (1) conduct a broad assessment of DCAA‘s management environment and audit quality assurance structure, (2) evaluate DCAA actions to date to correct previously identified problems, and (3) identify potential legislative and other actions for improving DCAA effectiveness and independence. To achieve these objectives, GAO analyzed DCAA‘s mission, strategic plan, audit policies, and quality assurance program; conducted interviews; reviewed selected audits at DCAA offices; and analyzed legislative and other actions. What GAO Found: GAO found audit quality problems at DCAA offices nationwide, including compromise of auditor independence, insufficient audit testing, and inadequate planning and supervision. GAO‘s conclusions stem from a review of 69 audit assignments supporting contract award and administrative decisions; an assessment of DCAA‘s audit quality assurance structure, which found similar audit quality problems but gave satisfactory ratings to deficient audits; and DCAA‘s rescission of 80 problem audit reports. The rescinded audits supported decisions on pricing and contract awards and impacted the planning and reliability of hundreds of other DCAA audits, representing billions of dollars in DOD expenditures. GAO findings include the following. Table: Selected Details of Audits GAO Reviewed: Contractor: Research and development grantee; Audit: Billing system; Significant case study issues: * DCAA auditors spent 530 hours to support an audit of a nonexistent billing system and reported adequate system controls. * Instead, DCAA should have relied on the Single Audit Act report on the grantee‘s cash management system. DCAA agrees. Contractor: Combat systems; Audit: Billing system; Significant case study issues: * This was a new system and therefore high risk, but auditors deleted key audit steps related to contractor policies and internal controls over progress payments without explanation. * One auditor told GAO he did not perform detailed tests because ’the contractor would not appreciate it.“ * DCAA allowed the contractor 7 months to address 6 significant deficiencies, dropping 2 and downgrading the other 4. * DCAA rescinded this audit report following GAO‘s review. Contractor: Iraq reconstruction; Audit: Accounting system; Significant case study issues: * Contractor objected to draft report, which included 8 significant deficiencies in the accounting system. * Auditors dropped 5 significant deficiencies and downgraded 3 others to suggestions to improve without performing new work. * Supervisory auditors directed audit staff to delete some audit documents, generate others, and in one case, copy the signature of a prior supervisor onto new documents making it appear that the prior supervisor had approved a revised risk assessment. * Supervisory auditor who approved altered documents was later promoted to western region quality assurance manager, where he served as quality control check over thousands of audits. Source: GAO. [End of table] GAO found DCAA‘s management environment and quality assurance structure were based on a production-oriented mission that put DCAA in the role of facilitating DOD contracting without also protecting the public interest. DCAA has taken several positive steps. However, DOD and DCAA have not yet addressed fundamental weaknesses in DCAA‘s mission, strategic plan, metrics, audit approach, and human capital practices that had a detrimental effect on audit quality. To improve DCAA oversight, the DOD Comptroller requested Defense Business Board and ’tiger team“ reviews and established a DCAA Oversight Committee. In addition, in the short-term, Congress could provide DCAA with certain legislative protections and authorities similar to those available to IGs. In the longer term, Congress may wish to consider organizational changes to elevate DCAA to a component agency reporting to the Deputy Secretary or to establish an independent governmentwide contract audit agency. What GAO Recommends: GAO makes 17 recommendations to DOD and the DOD Inspector General (IG) to improve DCAA‘s management environment, audit quality, and oversight. GAO also discusses matters that Congress should consider to enhance the effectiveness and independence of DCAA contract audits. DOD and DOD IG generally agreed with GAO‘s recommendations, concurring with all but two. View [hyperlink, http://www.gao.gov/products/GAO-09-468] or key components. For more information, contact Gregory Kutz at (202) 512- 6722 or kutzg@gao.gov. [End of section] Contents: Letter: Background: Nationwide Audit Quality Problems Are Rooted in DCAA's Poor Management Environment: DCAA Has Made Progress, but Correcting Fundamental Problems in Agency Culture That Have Impacted Audit Quality Will Require Sustained Leadership: Legislative and Other Actions To Improve DCAA's Effectiveness and Independence: Conclusions: Recommendations for Executive Action: Matters for Congressional Consideration: Agency Comments and Our Evaluation: Appendix I: Internal Control System Audits Did Not Meet Professional Standards: Appendix II: DCAA Does Not Perform Sufficient Work to Identify and Collect Contractor Overpayments: Appendix III: Objectives, Scope, and Methodology: Appendix IV: Comments from the Department of Defense: Appendix V: Comments from the Department of Defense Inspector General: Tables: Table 1: Examples of DCAA Audit and Nonaudit Services: Table 2: Summary of Five Selected Internal Control Audits: Table 3: Summary of Five Selected Cost-Related Assignments: Table 4: Summary of Selected DCAA Audit Quality Review Results: Table 5: GAGAS Noncompliance on 37 Selected Audits of Contractor Controls: Table 6: Case Studies of Problem DCAA Cost-Related Assignments: Table 7: Summary of DCAA Audits Reviewed for GAGAS Compliance: Table 8: Summary of DCAA Cost-Related Assignments Reviewed: Figures: Figure 1: DCAA Opinions on Contractor Internal Control Systems Audits: Figure 2: Comparison of DOD Contract Obligations and DCAA Workforce for Fiscal Years 2002 through 2008: Figure 3: DCAA Questioned Costs and Amounts Sustained by Contracting Officers: Abbreviations: AICPA: American Institute of Certified Public Accountants: AT: AICPA Statements on Attestation Standards: AT&L: Acquisition, Technology, and Logistics: APO: Audit Policy and Oversight (a Defense Inspector General organization): AU: AICPA Statements on Auditing Standards: CAS: Cost Accounting Standards: CAM: Contract Audit Manual, also referred to as the DCAA Contract Audit Manual, or DCAM: CFO: Chief Financial Officer: COSO: Committee on Sponsoring Organizations: DBB: Defense Business Board: DCAA: Defense Contract Audit Agency: DCAM: DCAA Contract Audit Manual: DCMA: Defense Contract Management Agency: DFARS: Defense Federal Acquisition Regulation Supplement: DFAS: Defense Finance and Accounting Service: DFMR: Defense Financial Management Regulation: DOD: Department of Defense: DPAP: Director of Defense Procurement Policy: FAO: field audit office: FAR: Federal Acquisition Regulation: FLA: Financial Liaison Advisor: FMR: Financial Management Regulation, also referred to as the Defense Financial Management Regulation, or DFMR: GAGAS: generally accepted government auditing standards: GAS: Government Auditing Standards: GPRA: Government Performance and Results Act: GS: General Schedule: IG: Inspector General: IGDH: Inspector General, Defense Handbook: OIG: Office of Inspector General: PCIE: President's Council on Integrity and Efficiency, renamed the Council of the Inspectors General on Integrity and Efficiency: SAS: AICPA Statements on Auditing Standards: SES: Senior Executive Service: SIGIR: Special Inspector General for Iraq Reconstruction: SSAE: AICPA Statements on Standards for Attestation Engagements: USC: United States Code: USD: Under Secretary of Defense: [End of section] United States Government Accountability Office: Washington, DC 20548: September 23, 2009: The Honorable Joseph I. Lieberman: Chairman: The Honorable Susan M. Collins: Ranking Member: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable Claire C. McCaskill: Chairman: Subcommittee on Contracting Oversight: Committee on Homeland Security and Governmental Affairs: United States Senate: This report addresses audit quality problems and independence issues at the Defense Contract Audit Agency (DCAA). In a September 2008 hearing before the Committee, we testified[Footnote 1] that DCAA failed to meet professional audit standards at three locations in California. Specifically, we found that the audit documentation for 14 selected audits at two locations did not support reported opinions, that DCAA supervisors dropped findings and changed audit opinions without adequate audit evidence for their changes, and that sufficient audit work was not performed to support audit opinions and conclusions. Further, we found that contractor officials and the Department of Defense (DOD) contracting community improperly influenced the audit scope, conclusions, and opinions of several audits, including forward pricing audits at a third location--a serious independence issue. During our investigation, DCAA managers took actions against their staff at two locations that served to intimidate auditors and create an abusive work environment. For example, we learned of verbal admonishments, reassignments, and threats of disciplinary action against auditors who spoke with or contacted our investigators, DOD investigators, or DOD contracting officials. At the time of the September 2008 hearing, we were conducting a broad assessment of DCAA's management environment and audit quality assurance structure at DCAA offices nationwide. Given the evidence presented at this hearing, you requested that we expand our ongoing assessment. This report therefore presents (1) an assessment of DCAA's management environment and quality assurance structure; (2) an analysis of DCAA's corrective actions in response to our July 2008 report,[Footnote 2] the Under Secretary of Defense (Comptroller/Chief Financial Officer) [Footnote 3] "tiger team" review,[Footnote 4] and the Defense Business Board study;[Footnote 5] and (3) potential legislative and other actions that could improve DCAA's effectiveness and independence. To assess DCAA's overall management environment and quality assurance structure, we analyzed DCAA's mission statement and strategic plan, performance metrics, policies and audit guidance, and system of quality control. We also reviewed audit documentation for selected audits at certain field audit offices (FAO) in each of DCAA's five regions for compliance with generally accepted government auditing standards (GAGAS)[Footnote 6] and other applicable standards. We selected 37 audits of contractor internal control systems performed by seven geographically disperse DCAA field offices within the five DCAA regions during fiscal years 2004 through 2006.[Footnote 7] These were the most recently completed fiscal years at the time we initiated our audit. Our approach focused on DCAA offices that reported predominately adequate, or "clean," opinions on audits of contractor internal controls over cost accounting, billing, and cost estimating systems issued in fiscal years 2005 and 2006.[Footnote 8] We selected DCAA offices that report predominately adequate opinions on contractor systems and related internal controls because contracting officers rely on these opinions for three or more years to make decisions on pricing and contract awards, and payment. For example, audits of estimating system controls support negotiation of fair and reasonable prices.[Footnote 9] Also, the FAR requires contractors to have an adequate accounting system prior to award of a cost-reimbursable or other flexibly priced contract.[Footnote 10] Billing system internal control audit results support decisions to authorize contractors to submit invoices directly to DOD and other federal agency disbursing offices for payment without government review.[Footnote 11] In addition, DCAA uses the results of internal control audits to assess risk and plan the nature, extent, and timing of tests for other contractor audits and other assignments. When a contractor has received an adequate opinion on its systems and related controls, DCAA would assess the risk for subsequent internal control and cost-related audits as low and would perform less testing on these audits. Although our selection of the seven offices and 37 internal control audits was not statistical, it represented about 9 percent of the total 76 DCAA offices that issued audit reports on contractor internal controls and nearly 18 percent of the 40 offices that issued 8 or more reports on contractor internal controls during fiscal year 2006. Of the 37 internal control audits we reviewed, 32 reports were issued with adequate opinions and 5 reports were issued with inadequate-in-part opinions. At the same seven DCAA field offices, we selected an additional 32 paid voucher, overpayment, request for equitable adjustment, and incurred cost assignments that were completed during fiscal years 2004 through 2006 for review of supporting documentation to determine whether DCAA auditors were identifying and reporting contractor overpayments and billing errors.[Footnote 12] In total, we reviewed 69 DCAA audits and cost-related assignments.[Footnote 13] To address our second objective, we assessed the status and analyzed several key actions that DCAA initiated as a result of our earlier investigation, including changes in performance metrics and policy and procedural guidance, as well as DCAA efforts in response to DOD Comptroller/CFO and Defense Business Board[Footnote 14] recommendations. To achieve our third objective to identify potential legislative and other actions that could improve DCAA's effectiveness and independence, we considered DCAA's current role and responsibilities; the framework of statutory authority for auditor independence in the Inspector General Act of 1978, as amended;[Footnote 15] best practices of leading organizations that have made cultural and organizational transformations; our past work on DCAA organizational alternatives; GAGAS criteria for auditor integrity, objectivity, and independence; and GAO's Standards for Internal Control In the Federal Government[Footnote 16] on managerial leadership and oversight. Throughout our audit, we met with the DCAA Director and DCAA headquarters policy, quality assurance, and operations officials and DCAA region and FAO managers, supervisors, and auditors. We also met with DOD Office of Inspector General (OIG) auditors responsible for DCAA audit oversight and DOD OIG hotline office staff. We conducted this performance audit from August 2006 through December 2007, at which time we suspended this work to complete our investigation of hotline complaints regarding audits performed at three DCAA field offices. We resumed our work on this audit in October 2008 and performed additional work through mid-September 2009 to evaluate DCAA's quality assurance program during fiscal years 2007 and 2008, assess DCAA corrective actions on identified audit quality weaknesses, and consider legislative and organizational placement options. During our assessment of DCAA corrective actions and analysis of legislative and organizational placement options for DCAA, we met with the former DOD Comptroller/CFO to discuss plans for Office of Comptroller/CFO and Defense Business Board reviews, and we continued to meet with and obtain information from the new DOD Comptroller/CFO and his staff. We also met with Comptroller's new DCAA Oversight Committee, which includes the Auditors General of the Army, the Navy, and the Air Force; the DOD Director of Defense Procurement and Acquisition Policy; and the DOD Deputy General Counsel for Acquisition. We obtained DOD and DOD OIG comments on a draft of this report. DOD and DOD IG comments are summarized in the Agency Comments and Our Evaluation section of this report. DOD comments are reprinted in appendix IV and DOD OIG comments are reprinted in appendix V. We conducted our audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for findings and conclusions based on our audit objectives. We performed our investigative procedures in accordance with quality standards set forth by the Council of the Inspectors General on Integrity and Efficiency (formerly the President's Council on Integrity and Efficiency). A detailed discussion of our objectives, scope, and methodology is included in appendix III. Background: DOD contract management continues to be a high-risk area for the government.[Footnote 17] With hundreds of billions of taxpayer dollars at stake, strong controls are needed to provide reasonable assurance that contract funds are not lost to fraud, waste, abuse, and mismanagement. Downsizing of contract administration personnel during the 1990s coupled with increased contract spending since 2000 have exacerbated the risks associated with DOD contract management. Our work continues to identify significant problems with federal agency contract payments[Footnote 18] and contract management.[Footnote 19] DCAA is charged with a critical role in DOD contractor oversight by providing auditing, accounting, and financial advisory services in connection with the negotiation, administration, and settlement of contracts and subcontracts. DCAA also performs contract audit services and payment reviews for other federal agencies, as requested, on a fee- for-service basis. DCAA contract audit services are intended to be a key control to help assure that prices paid by the government for needed goods and services are fair and reasonable and that contractors are charging the government in accordance with applicable laws, regulations (e.g., Federal Acquisition Regulation (FAR) and Defense Federal Acquisition Supplement (DFARS), standards (e.g., Cost Accounting Standards (CAS)), and contract terms. DCAA is headed by a director who reports to the Under Secretary of Defense (Comptroller/CFO). DCAA's placement provides the DOD Comptroller/CFO with access to financial information on defense contracts and allows the Comptroller/CFO to make this information available to the Secretary and Deputy Secretary of Defense. In addition, it permits the Comptroller/CFO to elevate policy issues concerning the scope of DCAA's authority and level of resources. The DCAA Director is responsible for day-to-day management of DCAA, development of strategic plans, audit guidance and procedures, and the quality of DCAA's audit services. DCAA's Contract Audit Manual (CAM)[Footnote 20] prescribes the standards, policies, and techniques to be followed by DCAA personnel in conducting contract audits. DCAA emphasizes and supplements CAM guidance through policy memorandums and other written notices, as well as through training and oral communications. The IG Act gives the DOD IG broad responsibilities to provide policy direction for and to conduct, supervise, and coordinate audits and investigations in DOD and in contractor operations, if warranted. DOD IG duties pertaining to DCAA include (1) providing policy direction for all DOD audits; (2) investigating fraud, waste, and abuse uncovered as a result of audits; (3) monitoring and evaluating adherence by all DOD auditors to audit policies, procedures, and standards; and (4) requesting assistance as needed from other auditors in DOD. As part of its audit policy and oversight responsibilities, the DOD IG reviews DCAA's system of audit quality control on a 3-year basis that is intended to meet the requirements under GAGAS for a peer review. DCAA History and Organizational Structure: Audits of military contracts can be traced back to at least the World War I era. Initially, the various branches of the military had their own contract audit function and associated instructions and accounting rulings. Contractors and government personnel recognized the need for consistency in both contract administration and audit. The Navy and the Army Air Corps made the first attempt to perform joint audits in 1939. By December 1942, the Navy, the Army Air Corps, and the Ordnance Department had established audit coordination committees for selected areas where plants were producing different items under contracts for more than one service. On June 18, 1952, the three military services jointly issued a contract audit manual that later became the DCAA CAM. In May 1962, Secretary of Defense Robert S. McNamara instituted "Project 60" to examine the feasibility of centrally managing the field activities concerned with contract administration and audit.[Footnote 21] An outcome of this study was the decision to establish a single contract audit capability within DOD and DCAA was established on June 8, 1965.[Footnote 22] At that time, DCAA's mission to perform all necessary contract audits for DOD and provide accounting and financial advisory services regarding contracts and subcontracts to all DOD components responsible for procurement and contract administration was established. The former Deputy Comptroller of the Air Force was selected as the DCAA Director and the former Director of Contract Audit for the Navy, was selected as the Deputy Director. DCAA was placed under management control of the Under Secretary of Defense (Comptroller), where it remains today. DCAA consists of a headquarters office at Fort Belvoir, Virginia, and six major organizational components--a field detachment office, which handles audits of classified contracting activity, and five regional offices within the United States. The regional offices manage field audit offices (FAO), which are identified as branch offices, resident offices, or suboffices. Resident offices are located at larger contractor facilities in order to facilitate DCAA audit work. In addition, regional office directors can establish suboffices as extensions of FAOs to provide contract audit services more economically. A suboffice depends on its parent FAO for release of audit reports and other administrative support. In total, there are more than 300 FAOs and suboffices throughout the United States and overseas. During fiscal year 2008, DCAA employed about 3,600 auditors at more than 300 FAOs throughout the United States, Europe, the Middle East, and in the Pacific to perform audits and provide nonaudit services in support of contract negotiations related to approximately 10,000 contractors. DCAA Audit and Nonaudit Services: DCAA's mission encompasses both audit and nonaudit services in support of DOD contracting and contract payment functions. FAR subpart 42.1, "Contract Audit Services," and DOD Directive 5105.36, Defense Contract Audit Agency (DCAA), establish DCAA as the department's contract audit agency[Footnote 23] and set forth DCAA's responsibilities. FAR 42.101 prescribes contract audit responsibilities as submitting information and advice to the requesting activity, based on the analysis of contractor financial and accounting records or other related data as to the acceptability of the contractors' incurred and estimated costs; reviewing the financial and accounting aspects of contractor cost control systems; and, performing other analyses and reviews that require access to contractor financial and accounting records supporting proposed and incurred costs. DOD Directive 5150.36 lists several responsibilities and functions that shall be performed by the DCAA Director,[Footnote 24] including: * "Assist in achieving the objective of prudent contracting by providing DOD officials responsible for procurement and contract administration[Footnote 25] with financial information and advice on proposed or existing contracts and contractors, as appropriate." * "Audit, examine, and/or review contractors' and subcontractors' accounts, records, documents, and other evidence; systems of internal control; [and] accounting, costing, and general business practices and procedures; to the extent and in whatever manner is considered necessary to permit proper performance of other functions —." These other functions cover contract audit and nonaudit services. In addition, the Directive states that the DCAA Director shall perform such other functions as may be assigned by the Secretary and Deputy Secretary of Defense or the Under Secretary of Defense (Comptroller/ CFO). * "Approve, suspend, or disapprove costs on reimbursement vouchers received directly from contractors, under cost-type contracts, transmitting the vouchers to the cognizant Disbursing Officer." DCAA uses the term audit to refer to a variety of evaluations of various types of data.[Footnote 26] In fiscal year 2008, DCAA reported that over 97 percent of its service work hours were spent on audits, meaning that DCAA has opted to provide nearly all of its services to the contracting and finance communities under applicable auditing standards, as discussed below. Table 1 lists several audit and nonaudit services provided by DCAA during the three phases of the contracting process--pre-award, contract administration, and close-out--and cites the statutory and regulatory provisions that authorize or establish the need to have DCAA perform the service. DCAA audits also support the contract payment process both directly and indirectly. For example, audits of contractor incurred cost claims and voucher reviews directly support the contract payment process by providing the information necessary to certify payment of claimed costs. [Footnote 27] Other audits of contractor systems, including audits of contractor internal controls, CAS compliance, and defective pricing, indirectly support the payment process by providing assurance about contractor controls over cost accounting, cost estimating, purchases, and billings that the agency may rely upon when making contract decisions, such as determinations of reasonable and fair prices on negotiated contracts. For example, an accounting system deemed to be adequate by a DCAA audit permits progress payments based on costs to be made without further audit.[Footnote 28] Table 1: Examples of DCAA Audit and Nonaudit Services: Pre-award phase: Contract phase and assignment: Accounting system[A]; Audit and Nonaudit services: Audit: DCAA determines adequacy of the contractor's accounting system prior to award of a cost-reimbursable or other flexibly priced contract. FAR 16.301-3(a)(1); Contracting support: [Check]; Payment support: Direct: [Empty]; Payment support: Indirect: [Check]. Contract phase and assignment: Contractor accounting disclosure statements; Audit and Nonaudit services: Audit: DCAA reviews the contractor's Disclosure Statement for adequacy and CAS compliance and determines whether the contractor's Disclosure Statement is current, accurate, and complete. DCAA also reviews Disclosure Statements during the post award phase if contractors revise them. FAR 30.202-6(c), 30.202-7 and 30.601(c); Contracting support: [Check]; Payment support: Direct: [Empty]; Payment support: Indirect: [Check]. Contract phase and assignment: Estimating system [A]; Audit and Nonaudit services: Audit: DCAA determines adequacy of contractor estimating systems. FAR 15.407-5 and DFARS 252.215- 7002(d),(e); Contracting support: [Check]; Payment support: Direct: [Empty]; Payment support: Indirect: [Check]. Contract phase and assignment: Contract price proposals and forward pricing proposals[B]; Audit and Nonaudit services: Audit: DCAA examines contractor records to ensure that cost or pricing data is accurate, current, and complete and supports the determination of fair and reasonable prices. 10 U.S.C. 2306a and 2313 (DOD) and 41 U.S.C. 254d (other agencies); FAR Subpart 15.4 (esp. FAR 15.404-2(c)) and 52.215-2(c); and DFARS 215.404-1; Contracting support: [Check]; Payment support: Direct: [Empty]; Payment support: Indirect: [Check]. Contract phase and assignment: Financial liaison advisory services[B]; Audit and Nonaudit services: Nonaudit: DCAA Director establishes and maintains liaison auditors and financial advisors, as appropriate, at major procuring and contract administration offices. These services are also provided during the post-award phase, as needed. DODD 5105.36, paras. 7.1.1 and 5.9; Contracting support: [Check]; Payment support: Direct: [Empty]; Payment support: Indirect: [Check]. Post award/administration phase: Contract phase and assignment: Internal control system audits (generally); Audit and Nonaudit services: Audit: DCAA reviews the financial and accounting aspects of the contractor's cost control systems, including the contractor's internal control systems. FAR 42.101(a)(3) and DFARS 242.7501; Contracting support: [Check]; Payment support: Direct: [Empty]; Payment support: Indirect: [Check]. Contract phase and assignment: Billing system audits[A]; DCAA reviews the financial and Audit: DCAA determines adequacy of contractors' billing system controls and reviews accuracy of paid vouchers. DCAA uses audit results to support approval of contractors to participate in the direct-bill program. FAR 42.101 and DFARS 42.803 (b)(i)(C); Contracting support: [Check]; Payment support: [Check]; Payment support: [Empty]. Contract phase and assignment: Purchasing system review[B]; Audit and Nonaudit services: Audit: DCAA determines adequacy of a contractor's or subcontractor's purchasing system. FAR Subpart 44.3; Contracting support: [Check]; Payment support: Direct: [Empty]; Payment support: Indirect: [Check]. Contract phase and assignment: Progress payments[B]; Audit and Nonaudit services: Audit: DCAA verifies amount claimed, determines allowability of contractor requests for cost-based progress payments, and determines if the payment will result in undue financial risk to the government. FAR 32.503-3, 32.503-4, and 52.232-16; Contracting support: [Check]; Payment support: [Check]; Payment support: [Empty]. Contract phase and assignment: Incurred cost claims[A]; Audit and Nonaudit services: Audit: DCAA determines acceptability of the contractors' claimed costs incurred and submitted by contractors for reimbursement under cost-reimbursable, fixed-price incentive, and other types of flexibly priced contracts and compliance with contract terms, FAR, and CAS, if applicable. FAR 42.101, 42.803(b), and DFARS 242.803; Contracting support: [Check]; Payment support: [Check]; Payment support: [Empty]. Contract phase and assignment: Billing rates and final indirect cost rates[A]; Audit and Nonaudit services: Audit: DCAA establishes billing rates for interim indirect costs and final indirect cost rates. FAR 42.704, 42.705 and 42.705-2 and DFARS 42.705-2; Contracting support: [Check]; Payment support: [Check]; Payment support: [Empty]. Contract phase and assignment: Defective pricing[B]; Audit and Nonaudit services: Audit: DCAA determines the amount of cost adjustments related to defective pricing. See above authorities to audit contractor cost and pricing data and FAR 15.407-1; Contracting support: [Check]; Payment support: [Empty]; Payment support: [Check]. Contract phase and assignment: CAS compliance[B]; Audit and Nonaudit services: Audit: DCAA determines contractor and subcontractor compliance with CAS set forth in 48 CFR 9903.201 and determines cost impacts of noncompliance. FAR 1.602-2, 30.202-7, and 30.601(C); Contracting support: [Check]; Payment support: [Check]; Payment support: [Empty]. Contract phase and assignment: Other specially requested services; Audit and Nonaudit services: Audit: Audit and nonaudit services: DCAA conducts performance audits and other audits based on requests from DOD components and requests from other federal agencies. DOD Directive 5105.36, Sec. 5; Contracting support: [Check]; Payment support: [Empty]; Payment support: [Check]. Contract phase and assignment: Paid voucher reviews[A]; Audit and Nonaudit services: Audit: Nonaudit services: DCAA reviews vouchers after payment to support continued contractor participation in the direct bill program. CAM 6-1007.6; FAR 42.803; DFARS 242.803; DODD 5105.36, paras. 5.4 and 5.5; and DOD Financial Management Regulation (FMR), vol. 10, ch. 10, para. 100202; Contracting support: [Check]; Payment support: [Check]; Payment support: [Empty]. Contract phase and assignment: Approval of vouchers prior to payment[A]; Audit and Nonaudit services: Audit: Nonaudit: DCAA reviews and approves contractor interim vouchers for payment and suspends payment of questionable costs. FAR 42.803; DFARS 242.803(b)(i)(B); DOD Directive 5105.36, paras. 5.4 and 5.5; and DOD FMR vol. 10, ch. 10, para. 100202; Contracting support: [Check]; Payment support: [Check]; Payment support: [Empty]. Contract phase and assignment: Overpayment reviews[A]; Audit and Nonaudit services: Audit: Non audit services: At the request of the contracting officer, DCAA reviews contractor data to identify potential contract overpayments. FAR 2.605, 52.216-7(g),(h)2; Contracting support: [Check]; Payment support: [Check]; Payment support: [Empty]. Close-out/termination phase: Contract phase and assignment: Contract close-out procedures and audits[A]; Audit and Nonaudit services: Audit: DCAA reviews final completion vouchers and the cumulative allowable cost worksheet and may review contract closing statements. DFARS 242.803(b)(i)(D); Contracting support: [Check]; Payment support: [Check]; Payment support: [Empty]. Source: GAO analysis. [A] Indicates DCAA audit and nonaudit services covered in this audit. [B] Indicates types of audits covered in our prior investigation (GAO- 08-857). We reviewed progress payment and contract close-out audits that related to audits in our earlier investigation or this audit where the auditors considered the evidence in those audits. [End of table] Importance of Audits in Accordance with GAGAS: DCAA policy states[Footnote 29] that it follows GAGAS[Footnote 30] when conducting audits. These standards provide a framework for conducting high quality government audits and attestation engagements. These standards also provide guidelines to help government auditors maintain competence, integrity, objectivity, and independence in their work and require that they obtain sufficient evidence to support audit conclusions and opinions. When auditors are required to follow GAGAS or are representing to others that they followed GAGAS, they should follow all applicable GAGAS requirements and should refer to compliance with GAGAS in the auditor's report.[Footnote 31] Most DCAA audits are performed as attestation audits under GAGAS. For attestation audits, GAGAS incorporates the American Institute of Certified Public Accountants (AICPA) general standard on criteria, and the field work and reporting standards and the related Statements on Standards for Attestation Engagements (SSAE), unless specifically excluded or modified by GAGAS.[Footnote 32] DCAA also conducts performance audits upon request. This report addresses DCAA attestation audits and related supporting assignments. GAGAS state that the public expects auditors to observe the principles of serving the public interest and maintaining the highest degree of integrity, objectivity, and independence in discharging their professional responsibilities. Serving the public interest and honoring the public trust are critical when performing government audits. Auditors increase public confidence when they conduct their work with an attitude that is objective, fact-based, nonpartisan, and non- ideological with regard to audited entities and users of the auditors' reports. Auditors also should be intellectually honest and free of conflicts of interest in discharging their professional responsibilities.[Footnote 33] Management of the audit organization sets the tone for ethical behavior throughout the organization by maintaining an ethical culture, clearly communicating acceptable behavior and expectations to each employee and creating an environment that reinforces and encourages ethical behavior throughout all levels of the organization. [Footnote 34] The credibility of auditing in the government sector is based on auditors' objectivity and integrity in discharging their professional responsibilities.[Footnote 35] Nationwide Audit Quality Problems Are Rooted in DCAA's Poor Management Environment: We found audit quality problems at DCAA offices nationwide, as demonstrated by serious quality problems in the 69 audits and cost- related assignments we reviewed, DCAA's ineffective audit quality assurance program, and DCAA's rescission of 80 audit reports in response to our work. [Footnote 36] Of the 69 audits and cost-related assignments we reviewed for this report, 65 exhibited serious GAGAS or other deficiencies similar to those found in our prior investigation, including compromise of auditor independence, insufficient audit testing, and inadequate planning and supervision. Although not as serious, the remaining four audits also had GAGAS compliance problems. The 69 audits and cost-related assignments we reviewed included 43 audits that DCAA reported were performed in accordance with GAGAS and 26 non-GAGAS cost-related assignments, including 10 overpayment and 16 paid voucher assignments. According to DCAA officials, DCAA rescinded the 80 audit reports because the audit evidence was outdated, insufficient, or inconsistent with reported conclusions and opinions and reliance on the reports for contracting decisions could pose a problem. Nearly one third (24) of the 80 rescinded reports relate to unsupported opinions on contractor internal controls, which were used as the basis for risk-assessments and planning on subsequent internal control and cost-related audits. Other rescinded reports relate to CAS compliance and contract pricing decisions. Because the conclusions and opinions in the rescinded reports were used to assess risk in planning subsequent audits, they impact the reliability of hundreds of other audits and contracting decisions covering billions of dollars in DOD expenditures. We found that DCAA's focus on a production-oriented mission led DCAA management to establish policies, procedures, and training that emphasized performing a large quantity of audits to support contracting decisions over audit quality. An ineffective quality assurance structure compounded this problem. Audit Quality Problems Found in All Audits GAO Reviewed: We found audit quality problems, including GAGAS compliance problems, with all 37 audits of contractor internal controls and the 4 incurred cost and the 2 request for equitable adjustment audits we reviewed at 7 FAOs across the 5 DCAA regions covered in this audit. In addition, none of the 26 cost-related assignments we reviewed from these same FAOs included sufficient testing to identify contractor overpayments and billing errors. For additional details on our analysis of these DCAA audits and assignments, including narrative case-studies, see appendixes I and II. Internal Control Audits: DCAA performs attestation audits of contractors' systems for cost accounting, estimating, and billing to gather evidence to express an opinion on the adequacy of the contractor's systems and related internal controls for compliance with applicable laws and regulations and contract terms. A contractor must have an adequate accounting system to be awarded a government cost-reimbursement contract, an adequate billing system to submit invoices for payment without government review, and an acceptable estimating system to support a contracting officer's approval of pricing proposals. A secondary objective of DCAA's audits of contractor systems and controls is to determine the degree of reliance that can be placed on the contractor's internal controls as a basis for planning the scope of other related audits. For example, if a contractor receives an adequate opinion on various systems control audits, auditors assess risk as low and reduce the level of testing on subsequent internal control and cost-related audits, including audits of contractors' annual incurred cost claims. Although the reports for all 37 audits of contractor internal controls that we reviewed stated that the audits were performed in accordance with GAGAS, we found GAGAS compliance issues with all of these audits. Examples of GAGAS compliance issues we found included: Independence issues. For 7 audits we reviewed, DCAA independence was compromised because auditors provided material nonaudit services to a contractor they later audited; experienced access to records problems that were not fully resolved; or significantly delayed report issuance in order to allow the contractors to resolve cited deficiencies. GAGAS state that auditors should be free from influences that restrict access to records or improperly modify audit scope.[Footnote 37] Insufficient evidence. We found that 33 of the 37 internal control audits did not include sufficient testing of internal controls to support auditor conclusions and opinions. GAGAS for examination-level attestation engagements require that sufficient evidence be obtained to provide a reasonable basis for the conclusion that is expressed in the report.[Footnote 38] However, our review of audit documentation often found that only two, three, or sometimes five transactions were tested to support audit conclusions, and the audit documentation did not contain a justification for the small sample sizes selected for testing. For internal control audits, which are relied on for 2 to 4 years and sometimes longer, the auditors would be expected to test a representative selection of transactions across the year and not transactions for just one day, one month, or a couple of months. [Footnote 39] For many controls, the procedures performed consisted of documenting the auditors' understanding of controls, and the auditors did not test the effectiveness of the implementation and operation of controls. Generally, the basis for an auditor's determination of sufficient testing should include (1) an adequate risk assessment, taking into consideration any auditor alerts arising from related audits, past findings, and corrective actions; (2) the contractor's overall control environment; and (3) the nature and volume of transactions and associated materiality and risk of error. For example, decisions on sufficient testing of contractor internal controls would include consideration of the number and types of contracts or proposals; the nature, dollar amount, and volume of transactions; and key control attributes or special characteristics of the transactions. Further, a representative selection would include a representative number of transactions from a population of transactions representing a reasonable period of time, in order for test results to support conclusions and opinions on the overall adequacy of the contractor's systems and effectiveness of the related controls. For example, under the GAO/PCIE Financial Audit Manual,[Footnote 40] the minimum sample size for an attribute sample of a control would be 45 items. Reporting problems. According to GAGAS, audit reports should, among other matters, identify the subject matter being reported and the criteria used to evaluate the subject matter. Criteria identify the required or desired state or expectation with respect to the program or operation and provide a context for evaluating evidence and understanding the findings.[Footnote 41] None of the 37 internal control audit reports we reviewed cited specific criteria used in individual audits. Instead, the reports uniformly used boilerplate language to state that DCAA audited for compliance with the "FAR, CAS, DFARS, and contract terms." As a result the user of the report does not know the specific Federal Acquisition Regulation (FAR), Cost Accounting Standards (CAS), or contract terms used as criteria to test contractor controls. This makes it difficult for users of the reports to determine whether a particular report provides the level of assurance needed to make contracting decisions. The lack of sufficient support for the audit opinions on 33 of the 37 internal control audits we reviewed rendered them unreliable for decision making on contract awards, direct billing privileges, the reliability of cost estimates, and reported direct cost and indirect cost rates. For example, the FAR requires[Footnote 42] government contracting officers to determine the adequacy of a contractor's accounting system before awarding a cost-reimbursement contract. Of the 9 audits of contractor accounting system internal controls that we reviewed, only two of the audits included sufficient testing to support DCAA's audit opinion that internal controls over the contractors' accounting systems were adequate. In addition, none of the 20 audits of contractor billing system internal controls we reviewed contained sufficient testing of controls to support the reported opinions. Adequate opinions on billing system audits are the basis for DCAA decisions to approve contractors for the direct bill program, whereby contractors submit invoices directly to a government disbursing office without prior review.[Footnote 43] Four of the 6 audits of contractor estimating system controls that we reviewed did not include sufficient testing to support the reported opinions. DOD requires[Footnote 44] that large contractors have acceptable estimating systems. Opinions on contractor estimating systems support DCAA decisions on the extent of testing performed on contract proposals. Neither of the two internal control audits of contractor indirect and other direct costs we reviewed included sufficient testing to support reported opinions. As shown in figure 1, at the time these audits were performed, DCAA policy guidance provided for three categories of opinions on internal control audits. This policy provided for different opinions and criteria for judging them based on the severity of the problems identified. Professional standards have long recognized different levels of severity with regard to reporting deficiencies and material weaknesses in internal controls. Figure 1: DCAA Opinions on Contractor Internal Control Systems Audits: [Refer to PDF for image: illustrated table] Risk: Low; DCAA opinion: Adequate; Criteria: No significant deficiencies were identified in the audit; Resultant Actions: Scope of future audits will be decreased based on assurance provided by adequate controls. Risk: Medium; DCAA opinion: Inadequate in part; Criteria: Auditors identified one or more significant deficiencies that affect parts of the contractor‘s system; Resultant Actions: Contractor is required to make improvements, and DCAA is to perform follow-up testing within 6 months. Inadequate in part opinion also requires expanded audit scopes on future and concurrent audits until the contractor‘s corrective actions are confirmed by the auditors. Risk: High; DCAA opinion: Inadequate; Criteria: Auditors identified one or more significant deficiencies that render the entire contractor system unreliable; Resultant Actions: Contractor is required to make improvements and DCAA is to perform follow-up testing within 6 months. Inadequate opinion requires expanded audit scopes on other audits because controls do not provide reasonable assurance that data generated by the contractor‘s system are reliable. Source: GAO analysis of DCAA policy. [End of figure] Supervisors of the DCAA internal control audits we reviewed dropped auditor findings of significant deficiencies from the audit reports or treated them as suggestions for improvement without adequate support, including instances of FAR noncompliance that should have been reported as material weaknesses. In some cases, auditors reported "inadequate-in part" opinions when the severity of the deficiencies or material weaknesses identified would have called for "inadequate" opinions. On December 19, 2008, DCAA revised its policy to eliminate the "inadequate-in-part" opinion and the requirement to report suggestions for improvement.[Footnote 45] The new DCAA policy defines "significant deficiency/material weakness" as an internal control deficiency that (1) adversely affects the contractor's ability to initiate, authorize, record, process or report government contract costs in accordance with applicable government contract laws and regulations; (2) results in a reasonable possibility that unallowable costs will be charged to the government; and (3) the potential unallowable cost is not clearly immaterial. The new DCAA policy also establishes new guidance on reporting audit opinions on contractors' internal control systems. For example, the new DCAA policy states that audit reports that identify any significant deficiencies/material weaknesses in contractors' internal control systems will include opinions that the systems are "inadequate." The policy notes that the contractor's failure to accomplish any control objective tested in DCAA's internal control audits will or could ultimately result in unallowable costs being charged to government contracts, even when the control objective does not have a direct relationship to charging costs to government contracts. As an example, the policy notes the control objective related to ethics and integrity is not directly related to charging costs to government contracts, but that the contractor's failure to accomplish the control objective creates an environment that could ultimately result in mischarging to government contracts. By eliminating the "inadequate-in-part" opinion, the new policy does not recognize different levels of severity and could unfairly penalize contractors whose systems have less severe deficiencies by giving them the same opinion--"inadequate"--as contractors having material weaknesses or serious deficiencies that in combination would constitute a material weakness. At the time we finalized our draft report for DOD comment, DCAA had rescinded 18 of the 33 audits of contractor internal controls that we determined did not contain sufficient testing to meet GAGAS.[Footnote 46] Unreliable audit opinions on contractor internal controls pose a significant risk because DCAA generally performs these audits on a 2-to 4-year cycle and the audit results are relied on for several years to make decisions on testing in various audits of contractor internal controls and cost-related assignments. In response to our earlier investigation in November 2008, DOD added DCAA audits not meeting professional standards to its list of material weaknesses.[Footnote 47] Table 2 provides details on five case studies that are typical of the flawed internal control audits that we reviewed during the course of our work. For more detail on the internal control audits we reviewed, see appendix I. Table 2: Summary of Five Selected Internal Control Audits: Case: 1; Region: Western; Audit type: Billing system (2004); Case details: * DCAA auditors inappropriately planned and performed a billing system audit of a federally funded research and development center (grantee) with $1.5 billion in annual funding. The grantee does not have a "billing system"; * The grantee is funded by a line of credit, which provides for cash draws and transaction reporting by the grantee's accounting system; * DCAA auditors spent 530 hours revising Single Audit Act cash management audit documentation to address procedures required in DCAA's standard audit program for billing system internal controls and developed a billing system audit report, when the auditors could have simply forwarded the results of work on the grantee's cash management system performed under the Single Audit Act to the federal agency's buying command; * As a result of our review, DCAA reassessed the need to perform a billing system audit for the grantee and determined that it would rely on the Single Audit reports in the future. Case: 2; Region: Western; Audit type: Accounting system (2004); Case details: * This audit involving accounting controls for one of the five largest DOD contractors working in Iraq was initiated in November 2003; * In September 2005, after nearly 2 years of audit work, DCAA provided draft findings and recommendations to the contractor that included 8 significant deficiencies in the contractor's accounting design and operation; * The contractor objected to the findings, stating that the auditors did not fully understand its new policies and procedures, which were just being developed for the fast track effort in Iraq; * Following the contractor's objections, various supervisory auditors directed the auditors to revise and delete some workpapers, generate new workpapers, and in one case, copy the signature of a prior supervisor onto new workpapers making it appear that the prior supervisor had approved a revised risk assessment; * On August 31, 2006, after dropping 5 significant deficiencies and downgrading 3 significant deficiencies to suggestions for improvement, DCAA reported an "adequate" opinion on the contractor's accounting system without adequate audit evidence for the changes; * The interim audit supervisor, who instructed the lead auditor to copy and paste the prior supervisor's name onto key risk assessment workpapers, was subsequently promoted to be the Western Region's quality assurance manager where he served as quality control check over thousands of audits, including those GAO reported on last year; * In April 2007, the Special IG for Iraq Reconstruction (SIGIR) reported that despite being paid $3 million to complete the renovation of a building in Iraq, the contractor's work led to plumbing failures and electrical fires in a building occupied by the Iraqi Civil Defense Directorate; * DCAA rescinded the audit report on December 2, 2008. Case: 3; Region: Eastern; Audit type: Billing system (2005); Case details: * In May 2005, DCAA reported an inadequate-in-part opinion on the billing system internal controls of a second of the five largest DOD contractors; * After issuing the report, DCAA auditors helped the contractor develop policies and procedures related to accounts receivable, overpayments, and system monitoring before performing a required follow-up audit--a serious impairment to auditor independence; * In June 2006, DCAA reported an adequate opinion on the contractor's billing system internal controls, including the policies and procedures DCAA helped the contractor develop; * As a result of GAO's review, DCAA rescinded the follow-up audit report on March 6, 2009. Case: 4; Region: Central; Audit type: Billing system; (2005); Case details: * This audit, which was initiated in July 2005, covered a new billing system at a business segment of another of the five largest DOD contractors. Although DCAA considers new systems to be high-risk and requires increased testing, auditors deleted key audit steps related to contractor policies and internal controls over progress payments from the standard audit program without explanation and performed little or no testing of the contractor's billing controls; * The contractor objected to requests for documentation to test whether billing clerks had received necessary training; * One auditor told GAO he did not perform other tests because "the contractor would not appreciate it"; * The auditors provided draft findings and recommendations to the contractor in February 2006 that included six suggestions to improve the system related to the need for internal audits, oversight of subcontractor accounting systems, and improvements in policies and procedures and desk instructions; * Instead of issuing the report, when audit work was completed and noting the status of any contractor actions to address identified control weaknesses, the auditors monitored contractor corrective actions for 7 months, dropping the two suggestions for improvement related to internal audits and monitoring subcontractor accounting systems. The failure to monitor subcontractor accounting systems should have been considered a significant deficiency; * On September 15, 2006, DCAA reported an "adequate" opinion on the contractor's billing system; * Following GAO's review of this audit, DCAA rescinded the audit report on February 10, 2009. Case: 5; Region: Central; Audit type: Billing system (2006); Case details: * A fraud investigation by the Army's Criminal Investigative Division was under way at the time DCAA performed this contractor's billing system audit. The FAO was aware of the substance of the Army's investigation; * The auditor requested increases in budgeted audit hours to perform increased testing because of fraud risk and the contractor's use of temporary accounts for charging costs that had not yet been authorized by the contracting officer; * The auditor drafted an "inadequate" opinion on the contractor's billing system, which was overturned by the supervisor and FAO manager; * Despite a reported $2.8 million in fraud for this contractor, DCAA reported an "inadequate-in-part" opinion related to 3 significant deficiencies in the contractor's billing system on August 31, 2005, and an "adequate" opinion on September 11, 2006, related to a follow-up audit; * The auditor, whose performance appraisal was lowered for performing too much testing and exceeding budgeted audit hours, was assigned to and then removed from the follow-up audit. The auditor left DCAA in March 2007; * Following GAO's review, DCAA rescinded both audit reports on November 20, 2008. Source: GAO analysis of DCAA audit documentation and auditor interviews. [End of table] Cost-Related Assignments: The 32 cost-related assignments we reviewed did not contain sufficient testing to provide reasonable assurance that overpayments and billing errors that might have occurred were identified. As a result, there is little assurance that any such errors, if they occurred, were corrected and that related improper contract payments, if any, were refunded or credited to the government. Contractors are responsible for ensuring that their billings reflect fair and reasonable prices and contain only allowable costs, and taxpayers expect DCAA to review these billings to provide reasonable assurance that the government is not paying more than it should for goods and services. Further, we found that DCAA does not consider some cost-related assignments to be GAGAS audits, even though these assignments are used to provide assurance of the reasonableness of contractor billings, for example: Paid voucher reviews. DCAA performs annual testing of paid vouchers (invoices) to determine if contractor voucher preparation procedures are adequate for continued contractor participation in the direct-bill program.[Footnote 48] Under the direct-bill program, contractors may submit their invoices directly to the DOD disbursing officer for payment without further review. Although DCAA does not consider its reviews of contractor paid vouchers to be GAGAS engagements; it has not determined what standards, if any, apply to these assignments. In addition, for the 16 paid voucher assignments we reviewed, we found that DCAA auditors failed to comply with CAM guidance.[Footnote 49] Rather than documenting the population of vouchers, preparing sampling plans, and testing a random (statistical) sample, auditors generally did not identify the population of vouchers, did not create sampling plans, and made a small, nonrepresentative selection of as few as one or two invoices for testing to support conclusions on their work. Even when DCAA auditors tested 20 or 30 invoices, they did not test billing controls or review supporting documentation for goods and services purchased. Instead, the auditors performed limited procedures such as determining whether the vouchers were mathematically correct and included current and cumulative billed amounts. Based on this limited work, the auditors concluded that controls over invoice preparation were sufficient to support approval of the contractors' direct billing privileges. However, the limited work performed does not provide assurance that contractor billings are accurate and comply with applicable laws, the FAR, CAS, and contract terms. This is of particular concern because we determined that Defense Finance and Accounting Service (DFAS) certifying officers rely on DCAA voucher reviews, and they do not repeat review procedures they believe to be performed by DCAA. Professional literature contains guidance to help auditors determine the level of testing that should be performed to obtain sufficient, appropriate evidence to support a conclusion that internal controls are effectively designed, implemented, and operating effectively. Inquiry alone does not provide sufficient, appropriate evidence to support a conclusion about the effectiveness of a control. Some of the factors that affect the risk associated with a control include: * the nature and materiality of misstatements that the control is intended to prevent, * the inherent risk associated with the related account(s) and assertion(s), * whether there have been changes in the volume or nature of transactions that might adversely affect control design or operating effectiveness, * the degree to which the control relies on the effectiveness of other controls (i.e., information technology controls), * the competence of personnel who perform the control or monitor its performance, and whether there have been changes in key personnel who perform the control or monitor performance, and: * whether the control relies on performance by an individual or is automated (an automated control would generally be expected to be lower risk if relevant IT general controls are effective).[Footnote 50] Professional standards[Footnote 51] state that the auditor should focus more attention on the areas of highest risk. As the risk associated with the control being tested increases, the evidence that the auditor should obtain increases. In addition, the GAO/PCIE Financial Audit Manual provides guidance on sampling control tests that would be relevant to DCAA testing of contractor invoices.[Footnote 52] The auditor should assess risk in determining the control attributes to be tested and select a sample that the auditor expects to be representative of the population. Attribute sampling requires random or systematic, if appropriate, selection of sample items without considering the transactions' dollar amount or other special characteristics. To determine the sample size, the auditor uses professional judgment to determine three factors--confidence level, [Footnote 53] tolerable rate (maximum rate of deviations from the prescribed control that the auditor is willing to accept without altering the preliminary control risk), and expected population deviation rate (expected error rate).[Footnote 54] Finally, the American Institute of Certified Public Accountants (AICPA) Audit and Accounting Guide: Audit Sampling[Footnote 55] (Audit Guide) contains attestation guidance on the application of SSAEs in specific circumstances, including engagements for entities in specialized industries. The Audit Guide states that an auditor using nonstatistical sampling is not required to compute the sample size using statistical theory. However, sample sizes of statistical and nonstatistical samples ordinarily would be comparable when the same sampling parameters are used.[Footnote 56] Overpayment assignments. DCAA intends these audits to verify that contractors have billing procedures and internal controls in place to identify and resolve contractor overpayments in a timely manner. DCAA guidance states that these engagements should be conducted in accordance with GAGAS to the extent applicable under the circumstances. [Footnote 57] However, none of the 10 overpayment assignments we reviewed were performed or reported as GAGAS engagements. We found that auditor judgments about the population and selection of transactions for these assignments did not provide a representative basis for testing and concluding on contractor controls over billings and payments received. For example, for the 10 assignments we reviewed, the auditors selectively reviewed an accounts receivable aging report to identify overpayments and determine if they had been resolved. The auditors did not attempt to identify the population of transactions subject to overpayments and over billings during the year, and they did not document their rationale for selecting a particular dollar threshold, number of transactions, or time period for testing contractor invoices. Our assessment of these assignments includes the same concerns regarding insufficient evidence to support the auditors' conclusions as discussed above for annual testing of paid vouchers. As a result, this work does not provide reasonable assurance that contractors have adequate controls in place to identify and correct overpayments and billing errors and make appropriate, timely refunds and adjustments. Incurred cost audits. The purpose of incurred cost audits is to examine contractors' cost representations and opine on whether the costs are allowable, allocable to government contracts, and reasonable in accordance with the contract and applicable government acquisition regulations.[Footnote 58] DCAA performs these audits as GAGAS attestation engagements. For the four incurred cost audits we reviewed, we found that the auditors did not adequately document their judgments about control risk or the sampling and test methodologies used. In addition, we found that the auditors traced claimed pool and base costs (indirect costs) to the contractor's accounting books and records to determine their accuracy and allowability. However, the auditors did not perform sufficient, detailed testing of support for claimed indirect and direct costs. The scope of work performed was not sufficient to identify claimed costs, if any, that were not adequately supported or unallowable costs, if any, that should have been questioned. In addition to the testing failures we identified on the 32 cost- related assignments, several additional issues came to our attention during our review: Exempting from professional standards certain assignments that were used as support for internal control system audits. We noted that paid voucher reviews and overpayment assignments, which were used to support direct-bill decisions and billing system audits, were not performed under GAGAS, even though some of them used the same terminology as GAGAS engagements to describe the work performed, including "comprehensive examination" and "audit." According to DCAA's CAM and DCAA officials, paid voucher reviews and most overpayment assignments are not intended to meet GAGAS standards. However, paid voucher reviews are intended to serve as audits of contractor payments, and DCAA's standard audit program for overpayment assignments states that the assignments are to be performed in accordance with GAGAS, unless there are specific exceptions. When these types of assignments are not conducted under professional standards, it is important for the report to clearly state the procedures performed and the intended uses of the report, such as verifying compliance with certain FAR requirements, in order to provide context for understanding the stated conclusions of the work and avoid misleading users of the report. Auditor objectivity issues. We also determined that DCAA's role with regard to making decisions to approve contractors for participation in the direct-bill program[Footnote 59] presented an impairment to auditor objectivity--which includes being independent in fact and appearance when providing audit and attestation engagements.[Footnote 60] The objectivity impairment relates to DCAA's audit role in authorizing contractors to participate in the direct-bill program, which places it in the position of making decisions that impact its nonaudit workload related to the review of contractor invoices prior to payment. For example, when contractors do not have direct billing privileges, DCAA acts as the authorized representative of the DOD contracting officer in reviewing contractor invoices prior to submission for payment. However, if DCAA auditors determine that a contractor has an adequate billing system, DCAA may authorize a contractor to participate in the direct- bill program, thereby eliminating workload related to review of the contractor's invoices prior to payment. In addition, the 20 billing system audits and follow-up audits we reviewed lacked sufficient testing to support reported opinions, or the opinions reported were inconsistent with the audit evidence. DCAA had approved all but 2 of the 16 contractors involved in these audits for the direct bill program. At the end of our audit, DCAA had not rescinded any of the memorandums or reports on the results of the cost-related assignments we reviewed. Table 3 provides details on five selected case studies of flawed cost- related assignments that we reviewed during the course of our work. Table 3: Summary of Five Selected Cost-Related Assignments: Case: 1; Region: Eastern; Type of assignment: Paid voucher review (2004); Non-GAGAS; Case details: * This contractor generates $1.1 billion in annual billings to the government; * The auditor assessed risk as low for this assignment without documenting the basis for the decision. The auditor then judgmentally selected 3 vouchers totaling $88,000 for testing out of a total of 222 vouchers submitted to the government for payment from March 2003 through February 2004; * The auditor tested the first voucher selected and performed limited testing on the remaining 2 vouchers. The workpapers do not include any evidence to show that the auditor performed most of the audit steps required in the standard audit program; * Despite limited testing, on March 31, 2004, DCAA prepared a Memorandum for the Record, stating "continued reliance can be placed on the contractor's procedures for the preparation of interim vouchers..." and "the contractor has met the criteria for continued participation in the direct billing program." Case: 2; Region: Mid-Atlantic; Type of assignment: Paid voucher review (2004); Non-GAGAS; Case details: * In 2004, DCAA reviewed interim vouchers submitted by a contractor with $40 million in annual sales; * The auditor chose a nonrepresentative selection of 3 vouchers totaling $621,000 from a 3-month period. The auditor should have used a population covering a 12-month period because this assignment was designed to cover a 1-year period; * The auditor did not document the sample selection methodology as required by DCAA's CAM. Although testing of 3 vouchers is not sufficient to support a conclusion on the effectiveness of the contractor's controls over preparation of interim vouchers, the auditor removed one of the 3 vouchers from testing and did not document a reason; * The auditor did not identify any errors in testing the two remaining vouchers; * On August 31, 2004, DCAA reported "continued reliance can be placed on the contractor's procedures for the preparation of interim vouchers" and "the contractor had met the criteria for continued participation in the direct billing program." Case: 3; Region: Western; Type of assignment: Paid voucher; review (2005); Non-GAGAS; Case details: * This DOD contractor with over $1 billion in annual billings to the government was one of several contractors that performed work to support the FBI's Trilogy investigative systems upgrade project; * The auditor tested less than 20 vouchers of 5,530 vouchers issued in a 12-month period; * On April 14, 2005, DCAA issued a Memorandum for the Record, stating "continued reliance can be placed on the contractor's procedures for the preparation of interim vouchers" and "the contractor has met the criteria for continued participation in the direct-bill program; * One year later, a GAO audit report revealed that during the time of this DCAA assignment, the contractor had over billed the FBI by over $400,000 in labor and improper first-class travel costs. Case: 4; Region: Central; Type of assignment: Overpayment assignment (2005); Non-GAGAS; Case details: * This DCAA assignment covered one of the five largest DOD contractors; * The auditor tested 4 transactions from a listing of potential underpayments and overpayments prepared by the contractor. The auditor did not independently verify the accuracy or completeness of the contractor's listing; * The audit program required the auditor to determine whether the contractor monitors the billings submitted by its top 3 to 5 subcontractors. However, the auditor performed this procedure for only 1 subcontractor based on "auditor judgment" and did not document the basis for this judgment in the audit documentation; * The auditor also relied on the unverified contractor-provided listing to identify refunds to the government. The auditor then "judgmentally selected" 2 refunds for testing from the contractor's listing; * The auditors' conclusions that the contractor's controls are sufficient to detect and correct billing errors and overpayments were not supported by sufficient testing or other independent evidence. Case: 5; Region: Western; Type of assignment: Incurred cost; audit (2004); GAGAS; Case details: * This audit covered a $516 million incurred cost claim submitted by a contactor performing reconstruction work in Iraq; * The auditors reported about $6 million in questioned costs and about $83 million in unsupported costs based on assist audits (portions of the audit performed by other FAOs) that had not been received by the report issue date; * Although the auditors charged 2,292 hours to this assignment, GAO determined that the auditors did not perform sufficient work to support the audit opinion. For example, the auditors traced claimed pool and base costs to the contractor's accounting books and records using a threshold of $5 million for cost-type contracts and $2 million for time and materials contracts, but did not perform detailed testing of support for transactions. Tracing amounts to the general ledger is not sufficient work to support an examination-level opinion and the auditors did not document the basis for the judgment used to determine the multimillion dollar thresholds; * Further, the auditors relied on testing performed in a related accounting system audit, which DCAA rescinded on December 2, 2008, in response to GAO concerns; * As a result, the auditor's risk assessment used to plan the incurred cost audit is no longer supported. Source: GAO analysis of DCAA audit documentation and auditor interviews. [End of table] We did not attempt to re-perform these assignments to find out whether actual overpayments or billing errors existed. For additional details on the cost-related assignments we reviewed, see appendix II. Poor Management Environment and Quality Assurance Structure at DCAA Impacted Audit Quality: We found that a management environment and agency culture that focused on facilitating the award of contracts and an ineffective audit quality assurance structure are at the root of the agencywide audit failures we identified. DCAA's mission and management goals focus on producing a large quantity of audits to support procurement and contract administration rather than assuring proper contract costs that help save taxpayer dollars. In addition, an ineffective audit quality control system and a "clean" peer review opinion compounded the problem, hindering DCAA management from identifying and correcting agencywide audit quality problems. DCAA's Mission Statement and Strategic Plan Do Not Focus on the Public Interest: DCAA's current mission statement does not address protecting the public interest in the manner in which it carries out audits to help assure that contractors charge fair and reasonable prices that comply with applicable laws and regulations, cost accounting standards, and contract terms. Instead, DCAA's mission statement calls for it to perform all necessary contract audits for DOD and provide accounting and financial advisory services regarding contracts and subcontracts to all DOD components responsible for procurement and contract administration. Similarly, DCAA's 2006 strategic plan focused on various processes and outputs. DCAA's strategic plan contains the following five strategic goals with targeted completion dates from 2006 through 2008: 1. fostering a quality work-life environment that promotes trust, teamwork, mutual respect, superior job performance and high morale; 2. assuring customer satisfaction by providing timely and responsible audits and financial services that meet or exceed customer requirements and expectations; 3. attaining the highest level of professional competence through continuous improvement in the management and performance of audits and services; 4. providing best value audit and financial services through continuous evaluation and improvement of audit and administrative processes; and: 5. providing an integrated information technology structure that promotes effectiveness and efficiency in providing services for internal and external customers. DCAA objectives under each strategic goal focus on process improvements and do not contain a clear plan for achieving the respective goal or adequate quantitative and qualitative measures for determining success, for example: * One DCAA quality of work-life objective is to assess whether the participative work team concept is the best model for facilitating continuous process improvement. The underlying activities include internal meetings and brainstorming sessions, literature reviews, and developing recommendations for executive committee review. None of the activities included refer to identifying best practices or working with outside experts. * Another objective is to hold or lower attrition in high turnover areas. DCAA activities in this area include analyzing causes of attrition, and conducting surveys of new hires and departing employees. None of the related activities include surveys of like organizations, consideration of best practices, or identifying and addressing causes of high attrition. Moreover, in response to our requests for attrition data, DCAA provided high-level summaries without any analysis. * DCAA's strategic goal for customer satisfaction, included the objective of increasing by 20 percent annually the number of incurred cost audit reports issued with contractor cumulative allowable cost worksheets, completing 100 percent of identified incurred cost audits necessary to accomplish Defense Contract Management Agency (DCMA) performance goals for contract close-out and canceling funds.[Footnote 61] DCAA's strategic plan contains no explanation of the importance of these objectives or how they link to DCAA's mission. * A key goal related to best value audit services is for DCAA to manage its cost per direct audit hour at a level sufficient to maintain DCAA's competitive advantage over the comparable national public firm composite rate. One of the ways DCAA has achieved a low cost per audit hour is to maintain a pay structure that caps journey-level auditors at the GS-12 level. In addition, our work identified numerous instances where entry-level auditors with little or no experience often perform audit assignments by themselves. However, lower grade levels and limited experience can place auditors at a disadvantage when dealing with contractor officials. The Government Performance and Results Act of 1993 (GPRA)[Footnote 62] directed federal agencies to shift their focus from traditional concerns of staffing and activity levels to a broad focus on outcomes or results by (1) defining a clear mission and desired outcomes instead of outputs, (2) measuring performance to gauge progress, and (3) using performance information as a basis for decision making. The act required agencies to meet with Congress and key stakeholders to clearly define their mission and develop long-term strategic goals as well as annual goals that were linked to them. Although these legislated requirements were directed at federal agencies, including DOD, DCAA's mission statement and strategic plan were not revised to conform to GPRA requirements. Performance Metrics Were Designed To Measure Output: GPRA also requires that once federal agencies establish their strategic goals they are to develop results-oriented measures for assessing performance in meeting those goals and publicly report on how well they are doing. However, most of DCAA's performance metrics continued to focus on output. Several DCAA managers noted that fear of outsourcing the contract audit function led DCAA to emphasize performance metrics that demonstrated high productivity and low cost. In fiscal year 2008, DCAA reported some results-oriented performance measures, such as return on investment and net savings related to questioned cost. However, most of DCAA's metrics focused on production and audit cost, including cost per direct audit hour, 30-day cycle time on forward pricing audits, and dollars audited per hour. In addition, DCAA's focus on completing over 30,000 assignments annually with about 3,600 auditors continued to emphasize production of audits instead of performing quality audits that assured taxpayers that the government was paying fair and reasonable prices for contracted goods and services. DCAA's Audit Quality Assurance Program Was Ineffective: DCAA's audit quality assurance program was not properly implemented, resulting in an ineffective quality control process that accepted audits with significant deficiencies and noncompliance with GAGAS and DCAA policy. Moreover, even when DCAA's quality assurance documentation showed evidence of serious deficiencies within individual offices, those offices were given satisfactory ratings. GAGAS require that each audit organization performing audits and attestation engagements in accordance with GAGAS should have a system of quality control that is designed to provide the audit organization with reasonable assurance that the organization and its personnel comply with professional standards and applicable legal and regulatory requirements, and have an external peer review at least once every 3 years.[Footnote 63] Our analysis of DCAA audit quality review documentation for 14 of 48 offices covered in audit quality reviews during fiscal years 2004 through 2006--the period covered in the last DOD OIG peer review--found that although DCAA gave satisfactory ratings to 13 of the 14 FAOs, DCAA reviewers reported that 10 of these offices had 2 or more instances of serious GAGAS noncompliance, including inadequate planning, lack of proper supervision, and insufficient support for reported conclusions and opinions. However, DCAA gave only 1 of the 14 FAOs reviewed an unsatisfactory rating. The failed FAO had 5 of 9 assignments reviewed with at least two significant instances of noncompliance with GAGAS or DCAA policy. Further, although DCAA headquarters performed a follow-up review to confirm that problems identified at the failed office were corrected, DCAA headquarters officials told us they did not perform follow-up reviews to assure that the problems identified at other offices were corrected. In response to a DOD IG finding that DCAA quality assurance reviews did not cover a sufficient number of internal control system audits, DCAA increased the number of audits covered to date in its fiscal year 2007 and 2008 quality assurance reviews. However, DCAA continued to inappropriately conclude that audits "demonstrated professional judgment," allowing reviewers to disregard serious deficiencies with GAGAS in concluding on overall audit quality.[Footnote 64] DCAA failed only 1 of the 40 FAOs as a result of its fiscal year 2007 and 2008 audit quality reviews. Our analysis of DCAA's audit quality results showed that 19 of the 40 FAO's had two or more audits with at least 2 instances of significant noncompliance with GAGAS or DCAA policy. However, 18 of these FAOs received a satisfactory rating. DCAA headquarters has not yet followed up with offices that had deficient audits. The examples in table 4 show the disparity between DCAA quality assurance reports of a "satisfactory level of compliance" and actual results documented by quality assurance reviewers. The examples below also illustrate the long-term nature of this problem. Table 4: Summary of Selected DCAA Audit Quality Review Results: Region: Eastern; Number and type of audits: 5 incurred cost audits; DCAA audit quality review conclusions and findings: On October 28, 2008, DCAA reported a satisfactory level of compliance for the FAO reviewed. Supporting documentation showed that reviewers found that 2 of 5 audits reviewed had at least 2 instances of significant noncompliance with GAGAS and DCAA policy, including insufficient supervisory involvement and inadequate workpaper documentation to support significant auditor judgments and conclusions. Region: North-eastern; Number and type of audits: 8 forward pricing audits; DCAA audit quality review conclusions and findings: On September 27, 2007, DCAA reported satisfactory compliance by the FAO reviewed. Supporting documentation showed that reviewers found that 4 of 8 audits had at least 2 significant instances of noncompliance with GAGAS or DCAA policy and 2 of the 4 audits had 3 instances of noncompliance, including inadequate planning and supervision and failure to exercise reasonable professional judgment. Region: Central; Number and type of audits: 8 other (various) assignments; DCAA audit quality review conclusions and findings: On April 4, 2006, reviewers gave the FAO reviewed a satisfactory rating. However, supporting documentation showed that audit quality reviewers found that 2 of 8 assignments had at least two significant deficiencies related to noncompliance with GAGAS and DCAA policy, including inadequate planning on 3 assignments and inadequate supervision on 2 assignments. Reviewers also determined that the auditor on one other assignment had not met the annual requirement for continuing professional education. Region: Western; Number and type of audits: 5 incurred cost audits; DCAA audit quality review conclusions and findings: On April 26, 2005, reviewers gave the FAO a satisfactory rating. Although the audit quality review documentation identified only 1 audit that had at least 2 instances of significant deficiencies, the documentation noted limited testing and stated that statistical sampling was not used, as required. The reviewers also found that audit working papers did not support the conclusions in the audit report. The reviewers noted that insufficient supervisory involvement was responsible in part for the deficiencies found in the audit. Region: Mid-Atlantic; Number and type of audits: 6 incurred cost audits; DCAA audit quality review conclusions and findings: On September 29, 2005, reviewers reported a satisfactory level of compliance for this FAO. However, supporting documentation showed that 4 of the 6 audits had at least 2 significant deficiencies related to noncompliance with GAGAS and DCAA policy. For example, audit quality reviewers noted that the risk assessment for one assignment inappropriately stated the contractor's accounting system was adequate. In addition, reviewers stated that conclusions and opinions in reports for three audits were not based on sufficient evidence. Reviewers also noted that three audits had significant deficiencies, including insufficient testing, inadequate procedures to identify illegal acts and noncompliance with laws and regulations, and reporting problems. Reviewers also found inaccuracies in reporting on three audits and stated that reports on 2 of the audits should not have been issued and a reported qualification in the report for the third audit was worded incorrectly and implied that work had been performed when the related assist audits had not been completed. Region: Eastern; Number and type of audits: 6 internal control audits; DCAA audit quality review conclusions and findings: On June 8, 2004, DCAA reported satisfactory compliance by the FAO reviewed. However, supporting documentation showed that 2 of 6 audits reviewed had at least 2 instances of significant noncompliance with GAGAS and DCAA policy, including inadequate supervision, missing workpapers on the contractor's control environment, and insufficient and incomplete workpaper evidence to support conclusions in the audit reports. Region: Northeastern; Number and type of audits: 8 forward pricing audits; DCAA audit quality review conclusions and findings: On June 26, 2003, DCAA reviewers reported satisfactory level of compliance by the FAO reviewed. Audit quality review documentation showed that reviewers found that 6 of the 8 audits had at least 2 instances of significant GAGAS or DCAA policy noncompliance. For example, 2 audits were not adequately planned and 4 audits had inadequate supervisory involvement. In addition, supervisory review was performed 10 days after the report was issued on one audit, and audit work did not support the reported opinion on a second audit. Source: GAO analysis of DCAA documentation. [End of table] In March 2009, DCAA officials advised us that going forward, DCAA plans to report all audit quality review findings along with recommendations for corrective action and follow-up to assure that FAOs have taken appropriate corrective action. DOD IG Peer Review Opinion on DCAA's Audit Quality Control System Is Inconsistent with the Underlying Deficiencies Reported: The DOD IG reported an adequate ("clean") opinion on DCAA's most recent peer review results although the reported evidence indicated that numerous audits had serious deficiencies in audit quality.[Footnote 65] In conducting DOD's audit oversight review of DCAA audits for fiscal year 2006, DOD IG audit oversight reviewers considered the same results of DCAA's internal audit quality assurance reviews that we analyzed and reviewed numerous additional audits, which also identified significant GAGAS noncompliance as evidenced by DOD IG peer review findings and recommendations. Although the DOD IG report contained evidence of significant, systemic noncompliance with professional standards throughout DCAA audits that OIG staff reviewed, and the IG report included numerous findings and recommendations related to those issues, the DOD IG gave DCAA a "clean" peer review opinion,[Footnote 66] concluding that: "In our opinion, the DCAA system of quality control for audits and attestation engagements performed during the FY ended September 30, 2006, was designed in accordance with quality standards established by Government Auditing Standards (GAS). Further, the internal quality control system was operating effectively to provide reasonable assurance that DCAA personnel were following established policies, procedures, and applicable auditing standards. Accordingly, we have determined that the DCAA system of quality control used on audits and attestation engagements for the review period ended September 30, 2006, is adequate." The overall report conclusion in the DOD IG report is not consistent with the detailed observations in the report, which indicate numerous significant deficiencies in DCAA's system of quality control. Furthermore, based on DCAA's actions to rescind dozens of audit reports [Footnote 67] related to our prior investigation and this audit and our analysis of DCAA's internal audit quality review procedures and documentation--all of which relate to the period covered by the DOD IG peer review--we concluded that DCAA's quality control system for the period covered by the last DOD IG peer review was not effectively designed and implemented to provide assurance that DCAA and its personnel comply with professional standards. The DCAA audits performed during fiscal years 2007 and 2008 were performed under the same policy guidance and production-related performance metrics as the earlier audits and had the same types of GAGAS noncompliance, as indicated by DCAA's internal audit quality review findings for audit reports issued in fiscal years 2007 and 2008. DCAA Lacks a Risk-Based Audit Planning Approach: In the absence of a risk-based audit planning approach, DCAA has historically performed 30,000 to 40,000 audits annually to support contracting community decisions on contract awards, administration, and close-out using 3,000 to 4,000 auditors--an average of about 10 audit reports per year for each auditor. The large number of assignments has contributed to the production-oriented environment and widespread problems we have identified with audit quality. The failure to perform quality audits leaves government contracting officers and disbursing officers with inadequate information, ultimately putting taxpayers at risk of improper contract payments and fraud, waste, abuse, and mismanagement. GAO's Standards for Internal Control in the Federal Government[Footnote 68] require federal agency managers to identify and assess relevant risks the agency faces from external and internal sources associated with achieving agency objectives, such as those defined in strategic and annual performance plans developed under the GPRA. To do this, management needs to consider all significant interactions between the entity and other parties as well as internal factors at the agency and activity levels. The specific risk analysis methodology used can vary by agency because of differences in agency missions and the difficulty in qualitatively and quantitatively assigning risk levels. For example, DCAA would need to consider requirements in law and federal regulation to audit contractor cost, price, schedule, systems, and compliance with laws, regulations, cost accounting standards, and contract terms. DCAA also would need to consider risks associated with contractor activity and the materiality of contractor costs. Once risks have been identified, sound management controls require that they should be analyzed for their possible effect, and management should decide how to manage the risk and what actions should be taken. A risk-based audit approach would help identify and prioritize which audits are the most important or have the highest return on investment and determine what constitutes appropriate testing for various audit and nonaudit services. Basing future audit plans on historical DCAA audit hour data is problematic because DCAA has not yet determined the time and effort that would be needed to perform quality audits. For example, historical audit hour data do not accurately reflect either the time needed to complete a quality audit or the hours actually worked on various audits because many auditors performed limited procedures or they performed audit procedures on their own time to meet budgeted audit hour metrics. In addition, some audits may not be necessary. For example, we concluded that 3 of the 37 internal control audits that we reviewed were not necessary. For one of the three audits, DCAA could have relied on the audit of a grantee that was performed under the Single Audit Act.[Footnote 69] DCAA agreed with our conclusion. Two other unnecessary audits involved estimating systems of contractors that only have one contract with the government. Because contract proposals, which would be tested as part of the estimating system audit for these contractors, are separately audited when they are submitted, we questioned the need for separate estimating system audits for these contractors. DCAA officials told us they would reconsider the need for separate estimating system audits in such cases. Developing a risk-based audit approach that considers the risk of improper contract payments and available resources would also be a first step in determining the level of audit resources and training needed to accomplish effective contract audits. In addition, determining appropriate roles and responsibilities for nonaudit assignments would further clarify DCAA audit resource needs as well as needed job skills and funding for buying commands and DCMA. The most pervasive audit deficiency we identified was insufficient testing to support DCAA's reported conclusions and opinions. Limited audit testing was directly related to DCAA's goal of performing 30,000 or more audit assignments annually. Achieving a goal of performing quality audits will depend, in part, on appropriate guidance on testing coupled with adequate training and supervision. Quality audits will also be dependent upon contracting community support of a risk-based audit approach and an appropriate delegation of nonaudit contract administration activities and audit responsibilities among DCMA, buying commands, and DCAA. As noted above, DCAA provides nearly all of its services to the contracting and finance communities as GAGAS audits. However, a risk-based audit approach may require these communities to re-evaluate whether all such services should be provided as audits and whether DCAA, as an independent audit organization would perform any nonaudit services. DCAA Lacks Effective Human Capital Management: DCAA's deficiencies in audit quality are directly related to its human capital management. Effective, efficient contract audits and oversight are dependent on a workforce that has the required skills to meet organizational goals and perform quality audits that serve the public interest, especially the taxpayer. Both GAGAS and GAO's Internal Control Standards[Footnote 70] require that personnel possess and maintain a level of competence that allows them to accomplish their assigned duties. GAGAS specifically requires that the staff assigned to conduct audit or attestation engagements under GAGAS must collectively possess the technical knowledge, skills, and experience necessary to be competent for the type of work being performed before beginning that assignment.[Footnote 71] GAGAS also requires attestation engagements to be properly supervised.[Footnote 72] Accordingly, agency management has a responsibility to identify appropriate knowledge and skills needed for various jobs and provide needed training, as well as candid and constructive counseling, and performance appraisals. DCAA's human capital management practices of hiring auditors at the entry-level and assigning them to complex audits with little classroom training or on- the-job experience and minimal supervision have contributed to the audit problems we identified. Inadequate training and supervision. DCAA headquarters officials acknowledged that the agency could improve developmental training and that it does not have continuing training for DCAA auditors throughout their career, referred to by DCAA as life-cycle training. Given the complexity of contract audits and identified DCAA audit quality problems, timely and effective training and appropriate supervision are critical to achieving effective audits. Auditors also should understand the professional standards they are required to follow. In addition, we found that on-the-job-training and supervision, which are key components of developmental training, were not consistently provided to new auditors. On-the-job training for new auditors varied by supervisor and by DCAA field office. For example, we previously reported[Footnote 73] that one of the offices in our hotline investigation had addressed this training need by assigning one supervisor to oversee trainee auditors and assigning trainee auditors to senior auditors who provided them on-the-job training during a particular audit. However, we identified 13 CAS compliance audits at this same office to which trainee auditors were assigned with little or no training or supervision. In addition, documentation for one of the team performance awards that we recently obtained from this office contained evidence that some trainee auditors were immediately given an audit assignment to carry out on their own. The performance award documentation stated as an achievement that "new hires were purposely assigned their own assignments as early as deemed appropriate in order to instill in them early the concept that they are responsible for the planning and conduct of their assigned audits. The supervisory and senior auditor—made a conscious decision to do this to avoid dependency issues with the new auditors." Our discussions with auditors in DCAA's 5 regions provide anecdotal examples of the training problems we found. For example, one auditor told us that entry level training is a "one-size-fits-all" approach that does not provide the right training at the right time, while four auditors told us they were not given enough time to develop their skills. One consistent comment from auditors was that on-the-job training was key to auditor effectiveness, but DCAA provided little or no opportunity for new auditors to obtain this developmental experience. Several auditors told us that trainees in their offices are given assignments to do on their own and that while trainees may work with a senior level auditor, sometimes these senior auditors do not take a leadership role that would provide a learning experience for trainees. In addition, several auditors described DCAA's internal training courses as "good," but noted that the courses covered high- level conceptual and technical information and did not provide the detailed knowledge on how to apply this information when performing a particular contractor audit. Some FAO managers share this concern. Supervisors responsible for deficient audits identified in GAO's prior investigation were promoted. At the September 2008 hearing, Committee Members expressed concerns about DCAA promotions of supervisors who were responsible for improperly dropped audit findings, unsubstantiated changes in audit opinions, and abusive management actions against whistleblowers at locations covered in our investigation. Best practices of leading organizations making organizational and cultural changes include top leadership who set the direction, pace, and tone and provides a clear, consistent rationale that unites staff together behind a single mission. Agency management plays a key role by setting and maintaining the organization's ethical tone, providing guidance for proper behavior, removing temptations for unethical behavior, and providing discipline, when appropriate. Our review of GAO hotline allegations received since our investigation showed that meeting metrics related to producing reports within budgeted hours and planned time frames resulted in performance awards for auditors who performed deficient audits with little or no testing and lower performance ratings and personnel actions that resulted in downgrades and termination of auditors who did not meet these metrics. Further, our analysis of performance appraisals and performance award information for auditors and supervisors at the location in our investigation where supervisors had been promoted[Footnote 74] showed that the supervisory auditors responsible for deficient audits at this location were rewarded with high performance appraisals, cash awards, and promotions. We obtained performance evaluations and performance award documentation for auditors and supervisors involved with 12 audits that had serious deficiencies at the first location we investigated in our prior work. The DCAA Director told us that there are legal issues associated with holding employees, such as the supervisory auditors, accountable for actions that were identified after-the-fact. However, the two supervisory auditors responsible for the deficient audits were approved for promotion even though Western Region managers who made promotion decisions were aware of the GAGAS compliance problems. DCAA's Western Region management had received the DOD IG's January 24, 2007, memorandum of investigation covering 10 audits performed at this location that did not meet GAGAS. Further, during the summer of 2007, Region management was responding to issues identified in our hotline investigation, which mirrored the IG's concerns and raised concerns about GAGAS compliance with four additional audits. Despite these findings, we found no evidence that supervisors and auditors who did not follow GAGAS and DCAA policy were disciplined, counseled, or required to take additional training. Instead, our review of performance appraisal and awards documentation showed that the supervisors and auditors responsible for the deficient audits received performance appraisals ranging from "exceeds fully successful" to "outstanding" along with numerous cash awards. One of the two supervisors responsible for inappropriate decisions to drop audit findings and change opinions without supporting evidence was promoted on October 14, 2007, and the second supervisor was selected for promotion on July 25, 2008--3 days after our investigative report was issued. DCAA placed a hold on the second supervisor's promotion pending further investigation. In addition, a senior auditor who dropped audit findings without support at the direction of the second supervisor was promoted to a supervisory auditor position on January 6, 2008. In contrast, the performance appraisal of the senior auditor witness from that office who testified at the Committee's September 2008 hearing was lowered two levels from "outstanding" to "fully successful" following the submission of her hotline complaint, and she received no cash awards. DCAA has rescinded all 12 audit reports and re-performed the 12 audits associated with our investigation at this field location. Allegations about abusive management actions have continued. We found that DCAA's current organization is highly decentralized, fostering a culture of region autonomy. Within this culture, DCAA's Western Region appears to have continuing problems with unresolved allegations of abusive management actions. For example, 21 of the 34 DCAA hotline allegations we received since our July 2008 report,[Footnote 75] include examples of abusive management actions, such as auditors being penalized for attempting to perform what they believe was sufficient testing to support audit opinions and auditors not completing work within established timeframes. Nine of these 21 allegations relate to DCAA's Western Region--the subject region in our prior hotline investigations. Seven of the 9 allegations relate to current problems in the Western Region. Our review of DCAA anonymous Web site contacts as of the end of May 2007 showed that over 40 percent (65 of 152) of the DCAA contacts also relate to the Western Region, including several that pertain to abusive management actions.[Footnote 76] Although DCAA headquarters officials have followed up on some of the complaints about management abuse that they received, decisions on disciplinary or corrective action typically have been delegated to region management. DCAA headquarters officials explained that in several cases, Western Region management has not agreed to take disciplinary or other available corrective actions. The officials told us that DCAA hotline staff have no recourse in these situations. DCAA Has Made Progress, but Correcting Fundamental Problems in Agency Culture That Have Impacted Audit Quality Will Require Sustained Leadership: Although DCAA has taken several positive steps, much more needs to be done to address widespread audit quality problems. DCAA's production- oriented culture is deeply imbedded and likely will take several years to change. Under the decentralized management environment, there has been little headquarters oversight of DCAA regions, as demonstrated by nationwide audit quality problems. Further, DCAA's culture has focused on hiring and promoting from within the agency and most training has been conducted by agency staff. This has led to a very insular culture where there are limited perspectives on how to make effective organizational changes. In response to our July 2008 investigative report,[Footnote 77] DOD's former Comptroller/CFO and Defense Business Board (DBB) conducted reviews[Footnote 78] of DCAA operations and made recommendations for corrective actions. The recommendations of the DBB are consistent with many of the recommendations discussed later in this report. DCAA has taken actions to revise performance metrics, change certain policy guidance, and obtain an independent organizational assessment (staff survey); however, DCAA has not yet addressed the fundamental weaknesses in its mission, strategic plan, audit approach, and human capital practices. Moreover, DCAA actions to date have focused on process and have not addressed the agency's decentralized organizational structure that has fostered a culture of DCAA region autonomy. On October 23, 2008, the DBB discussed its preliminary findings and recommendations at a public meeting, and on January 22, 2009, the DBB released its DCAA study report, which concluded that: * DCAA's mission focused primarily on supporting the procurement community with no mention of protecting taxpayer interest. The current mission statement also provided for advisory services that raised serious questions about DCAA's independence and objectivity under GAGAS. * DCAA's strategic plan did not address essential elements required by GPRA, and it did not address emerging issues that could affect mission accomplishment or contain a human capital strategic plan despite spending 80 percent of its budget on personnel. * None of DCAA's 24 performance measures addressed audit quality, such as conformance to GAGAS, and only 8 could be tied to DCAA's strategic plan. * DCAA's decentralized organizational culture dilutes effectiveness of managerial oversight and affects GAGAS compliance and audit quality. * DCAA has not established a human capital strategic plan as a key tool to facilitate human capital management and workforce development in support of DCAA's mission and implementation of its strategic plan. The following discussion summarizes the status of DCAA corrective actions on identified weaknesses, including actions on key DBB and DOD Comptroller/CFO recommendations. DCAA's Mission Statement and Strategic Plan Have Not Yet Been Revised: The DBB report, released in January 2009, pointed out that DCAA had five versions of a mission statement, noting that each version focused primarily on supporting the procurement community. The Board concluded that DCAA's mission should be refocused to protect the taxpayer's interests, writing: "The mission fostered the culture of supporting contracting officials, and the value system was one of quantity (number, cost, and timeliness of audits) over quality—which was further reinforced by the performance metrics that drove the organization." In addition, the Board reported that instead of complying with GPRA strategic planning requirements for long-term goals and objectives for major operations and functions, DCAA's plan resembled a short-term process improvement checklist and did not address enterprise risk, external factors, or emerging issues that could affect mission accomplishment. In addition, the Board noted that DCAA's strategic plan did not include an adequate human capital strategy to facilitate workforce development, recruiting, retention, and succession planning. The Secretary of Defense Has Not Yet Developed a DCAA Mission Statement That Focuses on Protecting the Public Interest: The DBB report recommended that the Secretary of Defense revise DCAA's mission to focus on protecting the interest of taxpayers, with the taxpayer as the primary customer, and that DCAA establish a core value of performing high quality, independent, and objective contract audits that adhere to GAGAS and ensure that taxpayer dollars are spent on fair and reasonable contract prices. The DBB did not address any amendments that might need to be made to the FAR, DFARS, and DOD Directives and policy documents that reflect DCAA's primary role as an advisor to government contracting officers and disbursing officers.[Footnote 79] Leading organizations that have undergone cultural and organizational transformation have identified top leadership involvement in developing a mission statement and strategic plan as a best practice. These organizations consider top leadership commitment in setting the direction, pace, and tone for the transformation as essential to provide a clear, consistent rationale that unites agency components behind the mission to guide the transformation. In meetings with DCAA officials, we expressed our concern that the Secretary of Defense had not taken action to revise DCAA's mission statement. On March 12, 2009, following a discussion on the preliminary results of our audit, the DCAA Director submitted a proposed revision to DCAA's mission statement to the Comptroller/CFO for review. The proposed revision inserted phrases that refer to "...serving the public interest" and "...ensure taxpayer dollars are spent on fair and reasonable contract prices." Although the revised mission statement had not been approved by the Secretary of Defense as of the end of July 2009, these changes would be positive. DCAA Has Not Yet Developed a Strategic Plan To Provide a Framework for Organizational and Cultural Reform: The DBB also recommended that DCAA develop a strategic plan that cascades from the revised mission statement and concurrently develop (1) an annual performance operating plan and a balanced scorecard tied to the strategic plan and (2) a human capital strategic plan. In addition, the DBB recommended that DCAA obtain an independent assessment of resource needs and engage an external professional organization to assist in a cultural transformation. DCAA officials told us they are having difficulty identifying an independent external professional organization to assist the agency in developing a strategic plan because DCAA audits most of the organizations that should be able to provide this type of assistance. In her February 27, 2009, response to the DBB report, the DCAA Director stated that DCAA expects to complete action on this recommendation by September 2009. With regard to the recommendation to develop a balanced score card, the DCAA Director reported that based on agreements with prior DOD Comptroller/CFOs, DCAA plans to use a monthly status report of agency performance measures rather than developing a balanced score card. Together with the change in performance measures for fiscal year 2009, DCAA implemented the monthly performance report in October 2008. The DCAA Director stated that DCAA will refine the annual performance plan in accordance with development of a revised strategic plan. The DCAA Director also stated that DCAA initiated a process to reengineer its human capital strategic plan in November 2008. The Director stated that DCAA obtained example plans from other organizations and attended training on human capital plan preparation and maintenance. DCAA is also seeking assistance from external organizations in reengineering its human capital plan. DCAA's Director Took Immediate Action To Eliminate Production Metrics, but Concerns about Audit Quality Measures Remain: The Committee's September 2008 DCAA oversight hearing raised concerns that DCAA's performance metrics focused on producing reports rather than performing quality audits and that auditors who attempted to perform quality audits were penalized for not meeting production goals. The DCAA Director acknowledged problems with the agency's metrics and stated that she had initiated a project to assess the agency's use of performance measures that would be completed by September 30, 2008. Performance metrics provide the basis for measuring achievement of agency mission and strategic goals. Accordingly, performance measures should be consistent with agency strategic goals. Although DCAA's mission statement and strategic plan have not yet been revised to provide new goals, the DCAA Director took action in September 2008 to eliminate production-oriented performance measures. On September 30, 2008, the Director issued a policy memorandum that eliminated 18 performance measures, identified 9 performance measures with goals for use in fiscal year 2009, and clarified the use and level of reporting on the revised measures. Some of the new performance metrics focus on outcomes, while others continue to focus on producing low cost audits in fixed time frames. New Performance Metrics Intended To Focus on Achieving Quality Audits: The DOD Comptroller/CFO required DCAA to develop standard metrics to measure and re-enforce compliance with GAGAS and CAM across DCAA by February 28, 2009. The DCAA Director reported that the new metrics established on September 30, 2008, met this requirement. DCAA identified the following six new performance metrics as focusing on the intended outcome-related goal of achieving quality audits that comply with GAGAS.[Footnote 80] 1. Obtaining an unqualified DOD IG peer review opinion. 2. DCAA's internal quality assurance program results show that 100 percent of the audits reviewed reflected professional judgment. 3. Checklist confirmation that issued reports did not include serious deficiencies. 4. A goal that 45 percent of audit reports will have findings as an indication of the tangible value of the audit work performed. 5. A goal that 15 percent of the audits will use quantitative methods to measure the extent to which advanced level audit techniques are used. 6. A goal that auditors will meet 100 percent of their continuing professional education requirements on time. Only metrics number 1, 3, and 6 have a direct relationship to audit quality. Although metric number 2 could improve audit quality if properly implemented, DCAA gave passing scores to deficient audits. Given the problems with DCAA's ineffective quality assurance program and DOD IG peer review results, for these three metrics to achieve the intended audit quality goal, significant changes will be needed in policy guidance and training on audit standards, appropriate procedures, and audit documentation in order to comply with GAGAS. The fourth goal that 45 percent of DCAA audit reports will have findings is approximately the same as the actual percentage of 41 percent of the reports in 2008. Because findings would support recommendations for corrective action, this metric could contribute to improvements in accountability over contractor cost and billings. Regardless of the goal, findings should be reported as appropriate based on the completion of quality audits. Further, the use of quantitative methods of analysis in audit reports needs to be supported by training on the appropriate methods for sampling and testing contract costs, controls, and compliance to help auditors perform sufficient testing to support audit conclusions and opinions. Performance Metrics That Continue To Measure Output: Although three of DCAA's fiscal year 2009 metrics are important in that they address responsiveness to contracting officer requests for audits, if not properly managed, they could impact the effectiveness of DCAA's new audit quality metrics. In the past, DCAA's efforts to meet contracting officer requests for audits within specified time frames caused auditors to sacrifice audit quality. The following three performance metrics continue to address issuing reports within specified times to support contract awards and closeouts. * A forward-pricing audit timeliness goal of 95 percent based on agreement with requesters. * Incurred cost audit timeliness goals of 90 percent of corporate audits completed within 12 months, 90 percent of major contractor audits completed in 15 months, and 95 percent of non-major contractor audits completed in 24 months. * An efficiency goal of cost per direct audit hour of less than $113.45 to be monitored at the agency level only. It is critical that agreements with the contracting community on timeliness goals for forward-pricing and incurred cost audits allow performance of sufficient audit procedures to help contracting officers ensure that prices paid by the government are fair and reasonable, and that contract costs comply with applicable laws, regulations, cost accounting standards, and contract terms. In addition, keeping cost per direct audit hour in line with past practices indicates that DCAA likely would continue to use trainee or junior auditors on assignments without senior auditor or supervisory auditor involvement. GAGAS requires that staff assigned to perform the audit or attestation engagement must collectively possess adequate professional competence for the tasks required.[Footnote 81] Moreover, DCAA has not agreed to develop a risk-based audit approach to address how it will perform required audits with available audit resources, reassess the need to perform 30,000 or more audits annually, and establish priorities for performing quality audits that meet GAGAS within available resources. On October 30, 2008, DCAA required regional audit managers to provide training on changes in performance metrics to all FAOs by December 31, 2008, as part of the effort to get the word out that DCAA's mission should be to protect taxpayer interest and that auditors should perform quality audits that meet GAGAS. The DCAA Director stressed to us that budgeted audit hours would be captured for planning purposes, but they were never intended and should not have been used to evaluate auditor performance. The DCAA Director also told us that DCAA auditor performance appraisals should not have considered exceeding budgeted audit hours as a performance failure. In addition, DCAA implemented an anonymous Web site for capturing feedback on inappropriate use of the new performance measures and abusive management actions. Inconsistent Implementation and Training on New Metrics: During random telephone calls made to 17 auditors across 15 FAOs in the five DCAA regions, we found mixed results on FAO implementation of DCAA's new performance metrics. DCAA's Assistant Director of Operations told us she also had become aware of some problems with regional audit managers meeting the requirement to provide training to FAOs on implementation of the new DCAA performance metrics. The Operations Assistant Director told us that she planned to follow-up with all FAOs in this regard. In response to our telephone calls, for example: * Auditors at 13 of the FAOs told us that metrics related to meeting budget hours for completing audits have been relaxed. Although most of these auditors were not aware of audit completion dates in fiscal year 2009 program plans for their offices, two auditors told us that audit completion dates had been pushed back to allow more time for performing individual audits. An auditor at a Northeast Region FAO told us the use of budget hours was flexible before the metrics changes, so there was no noticeable difference. An auditor at a Western Region FAO said that although budget hours are no longer a metric for individual auditor performance, there is still a lot of pressure on auditors to meet budgeted hours. * Auditors at 5 of the 15 FAOs told us they had received the mandatory training on changes in performance metrics prior to December 31, 2008. However, auditors at 4 FAOs told us they received the mandatory training after December 31, 2008, including two auditors at one Eastern Region FAO who told us they did not receive the required training until February 13, 2009. Auditors at the other 6 FAOs told us they received the metrics training, but they could not remember the dates of the training. * Auditors at 5 FAOs told us they were permitted to charge from 1 to 2 hours of administrative time per pay period for reading e-mails on DCAA policy changes and new policy memorandums. Auditors at 2 FAOs said they were not given an administrative code for this purpose. One of these auditors told us they read the policies on their own time. Auditors at the remaining 8 FAOs did not mention a time limit for reading DCAA policy memoranda. DCAA headquarters officials told us that auditors should be permitted to charge administrative codes for this purpose and that they are working to resolve this issue across DCAA. The DCAA Director advised DCAA employees that the new performance metrics would be revisited after 6 months to determine if changes are needed. On February 11, 2009, DCAA revised its fiscal year 2009 job objectives/performance plans to reflect the new performance measures, and DCAA's Deputy Assistant Director of Operations advised us that DCAA initiated an assessment of the new performance metrics in April 2009. DCAA Has Centralized, but Has Not Yet Restructured Its Audit Quality Assurance Program: DCAA has taken some actions to improve its quality assurance program. However, staffing difficulties and other issues have left the outcome of this important initiative uncertain. As previously discussed, GAGAS require that each audit organization performing audits and attestation engagements in accordance with GAGAS should have a system of quality control that is designed to provide the audit organization with reasonable assurance that the organization and its personnel comply with professional standards and applicable legal and regulatory requirements, and have an external peer review at least once every 3 years.[Footnote 82] In addition, considering the large number of DCAA audit reports issued annually and the reliance the contracting and finance communities have placed on DCAA audit conclusions and opinions, an effective quality assurance program is key to protecting the public interest. Such a program would report review findings along with recommendations for any needed corrective actions; provide training and additional policy guidance, as appropriate; and perform follow-up reviews to assure that corrective actions were taken. When we briefed DCAA on our preliminary findings in March 2009, DCAA had not yet taken action to correct serious deficiencies in its quality assurance program, including problems with DCAA's application of the professional judgment standard, whereby quality assurance program officials gave satisfactory ratings when significant noncompliance with GAGAS had been identified by reviewers. In response to our previous report, on August 20, 2008, the DOD Comptroller/CFO required that DCAA take certain actions to improve audit quality, which included a restructuring of DCAA's quality assurance function. Accordingly, on August 22, 2008, DCAA established a new headquarters Directorate for Quality Assurance and Integrity, which centralized the quality assurance function at DCAA headquarters. The DOD Comptroller/CFO directed that the new Quality Assurance Directorate be headed by a Senior Executive Service (SES) Deputy Director. Because DOD did not grant DCAA an additional SES position for this purpose, the DCAA Director assigned responsibility for leading DCAA's quality assurance function to a level GS-15, Assistant Director for Integrity and Quality Assurance. Under DCAA's management environment and culture, which continue to foster autonomous regions headed by SES-level directors, the grade level and experience of the GS-15 equivalent Assistant Director for Integrity and Quality Assurance pose a challenge when dealing with SES-level regional directors, deputy directors, and regional audit managers. For example, when presented with our findings and conclusions that various audits did not comply with GAGAS, DCAA headquarters policy and quality assurance managers allowed regions and FAO's to decide whether to rescind the subject audit reports. In March 2009, DCAA officials advised us that the GS-15, Assistant Director for Integrity and Quality Assurance position is an intended SES position and that two GS-15 Assistant Directors will perform as Chief of Integrity and Chief of Quality Assurance. In centralizing the quality assurance program, DCAA's new quality assurance organization provides for five GS-14 senior quality assurance auditors at DCAA headquarters and up to 27 GS-13 quality assurance auditors in the field assigned across the 5 DCAA regions. However, a headquarters requirement that all senior quality assurance staff relocate to DCAA headquarters at Fort Belvoir, Virginia, resulted in all but one of the five senior staff accepting other positions within DCAA because they did not wish to relocate. It took several months to recruit DCAA staff for the senior quality assurance positions at DCAA headquarters. On July 10, 2009, a DCAA headquarters official advised us that DCAA had selected staff to fill the two remaining vacancies, and these staff would be reporting for duty in the next few weeks. In response to our concerns that DCAA's quality assurance program has not resulted in audits that comply with GAGAS, DCAA officials advised us that going forward, DCAA will no longer rate an FAO's overall compliance with GAGAS and DOD policy. The officials told us that instead, DCAA headquarters plans to (1) report the detailed results of the audit quality reviews, (2) make recommendations to FAOs for any needed corrective actions, (3) conduct follow-up reviews for all FAOs with identified audit deficiencies to ensure that corrective actions are taken, and (4) provide training and policy guidance, as appropriate. If properly implemented, these procedures would help to assure an effective audit quality assurance program. DCAA Disagrees with the DBB Recommendation for a Risk-Based Audit Planning Process: The DBB recommended that DCAA establish a risk-based planning process that expands DCAA self-initiated audits and increases the potential for identifying fraud, waste, and abuse, and higher rates of return to the taxpayer by April 2009. The DBB intended for DCAA to audit any and all contracts awarded by the department. On February 27, 2009, in responding to the DBB recommendation, the DCAA Director stated that (1) DCAA's practice of auditing only certain contracts was due to regulation or statute and (2) absent the DCAA access-to-records clause in certain types of contracts, DCAA has no legal basis to obtain cost data from a contractor. The DCAA Director suggested that the DBB recommendation should be directed to the Under Secretary for Acquisition, Technology and Logistics, who oversees DCMA, and not DCAA. The DCAA Director told us that she believes that DCMA should address this recommendation because DCMA decides what audits DCAA should perform to support contracting decisions and DCMA would need to initiate action to change audit-related FAR requirements. Generally, DCAA, as the agent of the Secretary of Defense, has authority[Footnote 83] to examine records of (1) a contractor performing any cost-reimbursement, incentive, time-and-materials, labor- hour, or price re-determinable contracts and subcontractors performing such contracts of the contractor and (2) to evaluate the accuracy, completeness, and currency of certified cost or pricing data required to be submitted pursuant to law, all records of the contractor or subcontractor related to the proposal, and discussions conducted on the proposal, pricing of the contract or subcontract or performance of the contract or subcontract. This authority is implemented by insertion of the Audit of Records clause in solicitations for negotiated contracts. [Footnote 84] In addition, the Director of DCAA may require by subpoena the production of any records of a contractor that the Secretary of Defense is authorized to audit or examine. While DCAA does not have access to the records of all DOD contractors or statutory rights of access to contractor officials, we believe it has sufficient authority to undertake a risk-based audit approach consistent with its existing authority. Therefore, we believe the DBB recommendation for DCAA to develop a risk-based audit planning approach is appropriate. DOD acquisition officials we met with agree. Further, as previously discussed, a risk-based audit approach would provide a basis for determining audit resource requirements. DCAA has selected the Army Force Management Support Agency[Footnote 85] to perform its staffing study. However, DCAA is conducting a staffing study as a stand-alone effort rather than performing the study in concert with an effort to establish a risk-based planning process. To provide useful information for decision making, it is important that the staffing study and risk-based audit planning approach are conducted as integrated efforts. It is also important for the DOD contracting and finance communities to be involved in the staffing study analysis and planning process because, as discussed earlier, a risk-based audit approach may require these communities to re-evaluate whether all DCAA services should be provided as audits and whether DCAA, as an independent audit organization, should perform any nonaudit services. To address immediate staffing needs, DCAA requested funds for additional audit staff and training from the Defense Acquisition Workforce Development Fund,[Footnote 86] including 300 positions for fiscal year 2009 and another 200 positions in 2010. DCAA received approval of this request in December 2008. In May 2009, as part of DOD's request for an additional 9,000 positions for contract management and oversight, the DCAA request was increased from 500 to 700 new positions that are to be phased in from fiscal year 2009 through 2011.[Footnote 87] As previously discussed, without developing a risk- based audit approach, it is difficult to determine the level of resources needed to perform effective, quality contract audits. However, federal acquisition and contract audit resources have not kept pace with the growth on federal procurements. As shown in figure 2, although procurement obligations related to greater reliance on contractor-provided services and support of the Global War on Terrorism have more than doubled since fiscal year 2002, DCAA audit resources have remained about the same. In addition, contractor and subcontractor relationships have become more complex, increasing the complexity of contract audits. These changes underscore the need for a risk-based audit plan and assessment of auditor resource and training needs. Figure 2: Comparison of DOD Contract Obligations and DCAA Workforce for Fiscal Years 2002 through 2008: [Refer to PDF for image: multiple line graph] Fiscal year: 2002; DCAA audit workforce: 3,500; DOD contract obligations: $171 billion. Fiscal year: 2003; DCAA audit workforce: 3,500; DOD contract obligations: $195 billion. Fiscal year: 2004; DCAA audit workforce: 3,500; DOD contract obligations: $231 billion. Fiscal year: 2005; DCAA audit workforce: 3,500; DOD contract obligations: $270 billion. Fiscal year: 2006; DCAA audit workforce: 3,500; DOD contract obligations: $299 billion. Fiscal year: 2007; DCAA audit workforce: 3,600; DOD contract obligations: $331 billion. Fiscal year: 2008; DCAA audit workforce: 3,600; DOD contract obligations: $383 billion. Source: GAO analysis of unaudited obligations data from the Federal Procurement Data System and acquisition workforce data from the Office of Personnel Management. [End of figure] Although DCAA has undertaken certain initiatives to improve the effectiveness of audits of contractor billings and internal control systems, these efforts are not targeted for completion until September 2010 and September 2012, respectively, and they are not part of a comprehensive audit strategy or framework. Once decisions are made on changes in various audit procedures for these audits, related audit guidance and training would be needed to help ensure the new procedures are effectively implemented. DCAA Issued Revised Policy Guidance To Address Auditor Independence, Assure Management Involvement in Key Decisions, and Address Audit Quality Issues: Our investigation and audit identified problems and concerns related to auditor independence, the need for management involvement in key decisions, and audit quality. In response to our work, the DBB and DOD Comptroller/CFO made several recommendations for DCAA actions to address these concerns. Specific DCAA actions and our assessment include the following. Auditor independence. The DBB recommended that DCAA address advisory- type (nonaudit) services by (1) discontinuing participation on Integrated Product Teams and Source Selection Evaluation Boards, both of which impair auditor independence in fact and appearance under GAGAS; (2) reevaluating the role and number of Financial Liaison Advisors (FLA) to ensure independence and objectivity in both fact and appearance; and (3) working with the DOD acquisition leadership to explore alternatives for providing technical advice and support to the contract management community while adhering to the auditor independence standards in GAGAS. The DCAA Director responded that DCAA discontinued participation in Integrated Product Teams on August 4, 2008, and Source Selection Evaluation Boards on September 12, 2008. On November 23, 2008, DCAA realigned all FLAs to report directly to DCAA headquarters and completed an assessment of the number of advisors. DCAA is continuing to assess the functions performed by FLAs to assure their independence. The DCAA Director stated that if there is a significant change in the advisory functions, DCAA will initiate discussions with DOD acquisition leadership. We support efforts to reevaluate DCAA's nonaudit advisory services given the problems identified in our investigation. Although our review of DCAA's CAM guidance found that DCAA had established appropriate guidelines to avoid independence issues, we found that the auditors had not followed DCAA policy. According to the DCAA Director, the DBB's primary concern is that DCAA participation in these advisory services created the appearance of a lack of independence. Requirement for DCAA management involvement in key decisions. DCAA issued policy memorandums requiring that (1) FAO managers sign all audit reports issued by the FAO; (2) auditors elevate memorandums on disagreements with supervisors and FAO managers on draft audit opinions to the highest level necessary, including the DCAA Director, for resolution; and (3) auditors elevate problems in accessing contractor records to FAO managers, contracting officers, and regional offices for appropriate handling. DCAA action to require FAO managers to sign all audit reports issued by the FAO addresses concerns identified in our investigation that supervisors could inappropriately issue reports with adequate ("clean") opinions without review by FAO managers. Similarly, the policy to elevate disagreements on changes to audit opinions responds to findings in our investigation that supervisors ignored auditors' objections to dropped findings and changed opinions, and the auditors had no opportunity to elevate their disagreement beyond the supervisors. The access-to-records policy clarified actions required when auditors are denied access to records and required FAO managers to brief their staff on the revised guidance. The revised policy guidance emphasized that auditors (1) should follow procedures for addressing denial of access to records, which include notifying the FAO manager, contracting officer, and DCAA region; (2) take appropriate actions to effect a suspension or withholding of any unsupported costs billed to the government until the data are received and a determination is made regarding the allowability of the costs; and (3) question the unsupported costs in the audit report if the supporting documentation is not received prior to the completion of fieldwork. Although our work identified some access-to-records problems, in these cases, there was no evidence that DCAA supervisors elevated the issue to management or to procurement officials to initiate enforcement action, as set out in existing DCAA policy. Guidance to improve audit quality. On August 6, 2008, the DCAA Director requested that each FAO hold a stand-down day (where staff were relieved of assigned duties to take mandatory training) to discuss audit quality and the requirement to comply with GAGAS requirements for competence, integrity, objectivity, and independence in performing contract audits. In addition, DCAA issued policy guidance on adequate audit documentation and testing, including the following guidance that applied to assignments we reviewed for this report: * "Workpaper Documentation of Judgmental Selections"--requires a description of the universe (population) from which items are selected for testing, identification of items and attributes to be tested, and an explanation to support that the judgmental selection will result in adequate audit coverage. Emphasizing the requirement that audit documentation include a description of the population used for sampling and identification of items and attributes to be tested is appropriate. However, the requirement for an explanation in the audit documentation that the judgmental selection will result in adequate audit coverage needs to be sufficiently justified. GAGAS and AICPA standards require that auditors document significant decisions affecting the audit objectives, scope and methodology, findings, conclusions, and recommendations resulting from professional judgment.[Footnote 88] * "Audit Guidance for Annual Testing of Contractor Eligibility for Direct Bill," which is intended to determine whether continued reliance can be placed on the contractor's procedures for preparation of interim vouchers. This policy change clarified and consolidated audit steps related to the contractor's compliance with contract provisions, added audit steps for reviewing vouchers under time-and-material and labor- hour contracts, and removed the requirement to verify that the contractor's Central Contractor Registration is current. The policy memorandum states that this scope of work performed does not constitute an audit or attestation engagement under GAGAS. It is within DCAA's purview to determine whether these procedures constitute an audit. However, because direct-bill decisions present a risk of undetected improper contract payments, prudent decisions to continue a contractor's direct-bill authorization would necessarily be based on testing a statistical sample of invoices[Footnote 89] and include a review of supporting documentation, including documentation to confirm the government received goods and services noted on the billing invoice. We confirmed that Defense Finance and Accounting Service certifying officers rely on DCAA reviews, and they do not repeat review procedures they believe to be performed by DCAA. Human Capital Management and Cultural Transformation: The DBB made two recommendations to improve DCAA human capital management and agency culture, namely that DCAA (1) develop a human capital strategic plan as a key tool to facilitate human capital management and workforce development and (2) engage an external professional organization to assist in a cultural transformation that includes emphasizing core values such as quality, independence, ethics, and objectivity rather than a mindset focused on quantity and productivity. DCAA has not yet developed a human capital strategic plan as a key tool to facilitate human capital management and workforce development. In May 2009, DCAA finalized an agreement with the Naval Post Graduate School, Center for Defense Reform, for assistance on cultural reform. According to GAO's Internal Control Standards, [Footnote 90] operational success is possible only when the right personnel for the job are on board and are provided the right training, tools, structure, incentives, and responsibilities. Accordingly, management should ensure that skill needs are continually assessed and that the organization is able to obtain a workforce that has the required skills that match those necessary to achieve organizational goals. In addition, training should be aimed at developing and retaining employee skill levels to meet challenging organizational needs; qualified and continuous supervision should be provided to ensure that internal control objectives are achieved; and performance evaluation and feedback, supplemented by an effective reward system, should relate employee performance to the organization's success. Lack of a human capital strategic plan. The lack of a human capital management strategic plan has limited the effectiveness of DCAA's hiring, training, and staff development efforts. DCAA officials told us they view contract auditing as a highly specialized profession that requires knowledge of acquisition law and regulations and government procurement and contract management processes. As a result, DCAA officials believe that auditors must be hired at the entry level and trained to perform contract audits. The officials also believe that because DCAA is the only contract audit agency in the federal government and it operates the only federal contract audit training institute, DCAA is in the best position to train contract auditors. However, DCAA is not the only agency that performs contract audits. Many IG offices, including the DOD IG, the military service audit agencies, several executive agency IGs, and GAO all perform contract audits. Further, DCAA has not provided training that is designed to develop contract audit skills at successively higher levels, and it has not provided adequate or continuous supervision of audit staff. Moreover, our work has shown that performance evaluations and feedback have not always related performance to the agency's success, as was the case when supervisors who were responsible for improperly dropping audit findings and changing draft audit opinions received high performance evaluations and cash awards. At the September 2008 hearing, the DCAA Director acknowledged the need to develop revised training to address audit quality issues. However, it will take considerable time to develop a revised training program to address the range of changes in audit policies, processes, and procedures for performing quality audits in accordance with GAGAS. However, on April 8, 2009, DCAA revised its Supervisory Development Training Curriculum to emphasize leadership skills and better reflect the day-to-day activities performed by supervisors. This revision was based on feedback received through DCAA's suggestion program, anonymous Web site contacts, and focus groups and is not based on a study or expertise of an outside professional organization. In addition, DCAA has begun a reassessment of the 2-week technical indoctrination class for new hires. Although it is appropriate to consider staff input in developing training courses, the development of in-house training by agency personnel may not result in a design that encourages participants to develop more critical analysis of the underlying principles or ways to bring about organizational change. Outside expertise helps ensure that an organization benefits from outside subject matter experts as well as education and training professionals who have a broad perspective on innovative approaches to best practices or best learning design. DCAA has difficulty identifying an independent professional organization to assist in cultural transformation. According to the DCAA Director, DCAA faces challenges in engaging a professional organization to assist with cultural change because (1) many external organizations that provide this service are audited by DCAA and to preserve the appearance of independence under the auditing standards, DCAA cannot engage organizations that it audits and (2) based on initial discussions with various organizations, DCAA believes this effort could cost from $1 to $2 million or perhaps more and DCAA would need additional funding to pay for this assistance. However, based on an assumption that DCAA would receive funding for this effort, the Director established a completion date of January 2010 with training of the workforce potentially extending into fiscal years 2011 and 2012. In the face of these challenges, the DCAA Director took action on three other initiatives related to cultural change. The DCAA Director stated that shortly after issuance of our July 2008 report, DCAA initiated a 1- to 2-year project to accomplish an organizational assessment using the Baldrige National Quality Program[Footnote 91] criteria with assistance from Baldrige experts within the Army. In addition, as required by the DOD Comptroller/CFO in September 2008, the Director asked the Office of Personnel Management to conduct an independent organizational survey of DCAA. As previously discussed, to help ensure that DCAA's new performance metrics resulted in appropriate cultural change with regard to the new emphasis on audit quality, DCAA established an anonymous Web site for obtaining feedback on the inappropriate use of the performance measures. In May 2009, DCAA asked the Naval Postgraduate School, Center for Defense Reform to assist DCAA with cultural transformation as recommended by the DBB. The Center began work in June 2009 to help DCAA identify issues facing the organization and develop an action plan. Delay in reporting results of DCAA's organizational survey. DCAA's independent organizational survey was completed during the fall of 2008, and DCAA officials said the assessment results would be finalized in March 2009, but then amended the date for completing the assessment of the survey results to late July 2009. Therefore, the survey results were not available to us for review. DCAA's anonymous Web site contacts underscore the need for a separate hotline office. DCAA's anonymous Web site was established as a mechanism for monitoring compliance with DCAA's new performance metrics; however, it has become an internal hotline, with many auditors reporting the same issues as those presented in hotline complaints received by GAO. The DBB report stated that DCAA would benefit from the establishment of a Chief of Internal Review to perform critical inspector general functions, such as performing periodic reviews and evaluations, serving as an ombudsman between staff and DCAA management, and addressing hotline complaints. Instead of establishing a separate Internal Review function, in March of 2009, the DCAA Director divided responsibilities of its Operations Directorate between the Operations Assistant Director and Deputy Assistant Director to provide dedicated staff to handle auditor concerns reported to the internal DCAA anonymous Web site. DCAA's Assistant Director of Operations along with a Division Chief and three program managers were made responsible for the DCAA hotline function, and the Deputy Assistant Director of Operations was given responsibility for day-to-day operations. Our review of DCAA headquarters handling of DCAA auditor concerns and hotline allegations sent to DCAA's anonymous Web site determined that internally reported DCAA auditor concerns represent problems across all five DCAA regions. As with GAO hotline complaints, the largest number of problems reported to DCAA's anonymous Web site related to DCAA's Western Region. Our review of DCAA documentation and discussions with auditors and DCAA officials indicate that current handling of internally reported DCAA auditor concerns and allegations appears to be timely, objective, and fact-based. The Assistant Director of Operations has made good progress in establishing credibility and trust in the DCAA hotline function. It will be important for any future inspector general or ombudsman to carry forward in this role. The DCAA Director's response to the DBB report did not address the recommendation to establish a Chief of Internal Review. We agree with the DBB recommendation. It is important for DCAA to have a hotline function that is separate from management and operations. Currently, the Operations Assistant Director has been reassigned to handle this function on a temporary basis. However, given the size of the DCAA organization, the extensive number of internal DCAA hotline complaints- -which totaled about 150 at the end of May 2009--and the likelihood of continuing hotline contacts that would need to be addressed as DCAA undergoes its cultural transformation, a permanent internal review or inspector general function is warranted. Legislative and Other Actions To Improve DCAA's Effectiveness and Independence: In addition to correcting the fundamental weaknesses in mission and the overall management environment discussed above, certain legislative and other actions, such as changes in organizational placement, could enhance DCAA's effectiveness and independence. Successful management initiatives for cultural and organizational change in large private and public sector organizations can often take several years to accomplish. Changing DCAA's organizational placement without first correcting fundamental weaknesses in mission and the overall management environment would not assure effective audits. Given this time frame and pursuant to your request, we identified legislative and other actions that decision makers can consider to improve DCAA's effectiveness. In the short term, Congress could enhance DCAA's effectiveness and independence by granting DCAA certain authorities and protections similar to those offered to presidentially appointed inspectors general under the Inspector General Act of 1978, as amended [Footnote 92] (IG Act). The IG Act contains provisions that enhance the independence of presidentially appointed IGs, including protections from removal without congressional notification, access to independent legal counsel, public reporting of audit results, rights to take statements from contractor and other personnel, and budget visibility. These provisions would enhance the important DCAA initiatives currently under way. Continued monitoring and oversight will be essential to assuring the successful implementation of DCAA's management initiatives. In the longer term, Congress could consider changes in organizational placement after current reform efforts have been effectively implemented. However, moving DCAA as an organization would require careful analysis and planning before implementation. Short-Term Legislative Actions: In addition to DCAA management reforms already under way and our additional recommendations, we identified certain legislative protections and authorities under the IG Act that could enhance DCAA's effectiveness. Legislation would be needed in order to grant DCAA such protections and authorities. Leadership. The IG Act provides for the President to appoint the IG, with Senate confirmation, at many federal agencies.[Footnote 93] Under the act, Congress must be notified in advance of removing the IG, and only Congress can eliminate the office of an IG. Currently, the head of DCAA is appointed and can be removed by the Secretary of Defense. Further, DCAA was created and can be reorganized or reassigned by departmental order without notice. IG Act protections Congress could grant to DCAA would therefore include (1) Senate confirmation of a presidentially appointed DCAA Director[Footnote 94] and (2) removal of the DCAA Director conditioned on congressional notification. [Footnote 95] Specifically, the act provides that an inspector general may be removed from office by the President and any removal is to be reported to both Houses of Congress 30 days prior to the removal. In addition to these IG Act protections, Congress could build additional provisions into legislation, to include the following: * Requirements that the DCAA Director possess the appropriate professional qualifications. For example, provisions for appointment of the DCAA Director could require selection from among individuals who possess demonstrated ability in managing and leading organizations, specific accounting or auditing background, general knowledge of contract management, and knowledge of and extensive practical experience in financial management practices in large governmental or business entities. * A mandate permitting the DCAA Director to hold a renewable term appointment for between 5 to 7 years. Legislation should provide that the DCAA Director can be removed only for cause or other stated reasons. These protections would allow the head of DCAA to provide stability and continuity of leadership that span presidential administrations and prevent removal except for cause or other disclosed reasons. * Conflict of interest provisions for the DCAA Director and other key staff in addition to those provisions currently in law. This would be intended to ensure that selection of the audit agency head would not involve a "revolving door" situation between contractors and the contract audit agency. Access to independent legal counsel. The IG Act provides for independent legal advice for IGs rather than requiring the use of agency legal counsel.[Footnote 96] Currently, DCAA relies upon DOD legal counsel. DCAA officials told us that the DCAA Director has not always been apprised of legal decisions by DOD counsel that have impacted DCAA operations. Further, according to the DCAA Director, the lack of independent counsel led to a situation where DOD attorneys provided questionable legal counsel to a DCAA field office supervisor without the DCAA Director's knowledge. Obtaining independent legal counsel would avoid conflicts of interest between DOD and DCAA, thereby helping to improve DCAA's effectiveness. Budget. The IG Act requires separate budgets for Offices of Inspector General (OIG) within agency budgets, allowing Congress to review IG budget requests separately. DCAA currently does not have this protection. IGs that are appointed by the President with Senate confirmation receive a separate appropriation, preventing agencies from reprogramming IG funds to other programs and activities. However, there is currently little visibility of DCAA's budget because it is funded under the Operations and Maintenance, Defense-wide appropriation, which includes numerous DOD agencies, such as the Defense Contract Management Agency (DCMA), the Defense Logistics Agency, the Defense Finance and Accounting Service, and some buying command activities. Therefore, DCAA's share of annual appropriations is subject to reprogramming, sometimes without congressional notification. According to the DCAA Director and documentation provided by the Director and Office of Comptroller/CFO, in the past, DOD has reprogrammed funding between DCAA and other DOD activities on numerous occasions. Because these reprogrammings were below the $15 million threshold for congressional notification, Congress did not have notice of these funding decreases at the time they occurred. For fiscal year 2009, DOD reprogramming increased DCAA's funding by $3.5 million. Legislation similar to the IG Act could grant DCAA a separate budget[Footnote 97] to provide visibility and protections from reprogramming of funds to other agency priorities. Increased authority and independence. Legislation could strengthen DCAA's audit authority by providing the same level of access to records and personnel available to IGs.[Footnote 98] Currently, DCAA has statutory rights of access to certain records related to cost-type contracts or those that contain cost and pricing data, but not to contractor personnel. As a result, DCAA's subpoena power is limited to certain records and does not cover contractor personnel. While we recognize that DCAA auditors have ongoing discussions with contractor personnel, they do not have statutory authority to compel contractor officials to meet with them and submit to interviews. IGs have authority, including subpoena power, to access all records, reports, audits, reviews, documents, papers, recommendations, or other material available that relate to programs and operations for which the IG has responsibilities. Further, IG subpoena authority extends beyond access to records and documents in that IG auditors can administer or take an oath in order to obtain information. Our discussions with DCAA auditors and reviews of audit documentation identified numerous instances where requests for contractor records were not met.[Footnote 99] Obtaining increased access to contracting companies, especially their staff and documentation, would be an important provision to improve the effectiveness of DCAA audit staff. Reporting and oversight of audit results. The IG Act provides for semi- annual reports to the agency head and appropriate committees of Congress summarizing results of significant audits and investigations.[Footnote 100] DCAA currently has no external reporting requirement, reducing opportunities for oversight and transparency. Congress could mandate some form of external DCAA reporting in legislation similar to the IG Act. Moreover, DCAA does not currently provide copies of its audit reports to other federal agencies that use the same contractors that DOD uses. According to the DCAA Director, DCAA's appropriations are specific to DOD contractor audits, and unless federal agencies request and reimburse DCAA for audit services, DCAA cannot provide them with copies of its audit reports even though its DOD audits of systems and related internal controls, cost accounting system compliance, etc. may cover their contractors. Legislation could also expressly allow DCAA to provide audit results to other agencies, a step that would improve its visibility and effectiveness for the government as a whole. Legislation to grant DCAA similar protections and authorities as those provided in the IG Act could enhance reform efforts that are already under way. Although we found that a lack of DOD Comptroller/CFO and IG oversight has impaired DCAA's effectiveness, DOD has begun work to provide improved oversight of DCAA's operations. In August 2008, the DOD Comptroller/CFO conducted a "tiger team" review of DCAA's audit quality assurance program, and DOD approved a more comprehensive Defense Business Board (DBB) study. The new DOD Comptroller/CFO recognized the need for DCAA oversight and on March 16, 2009, approved the charter for a DCAA Oversight Committee. Committee members include the Auditors General of the Army, the Navy, and the Air Force; the DOD Director of Defense Procurement and Acquisition Policy; and the DOD Deputy General Counsel for Acquisition and Technology. The Committee held its first meeting in early April 2009. During May 2009, the DCAA oversight committee members reviewed selected DCAA audits and visited a DCAA field office. In addition, the committee members have indicated that they plan to review this report, our earlier investigative report, the DOD Comptroller/CFO "tiger team" report, the DBB report, and the upcoming DOD IG report that follows up on issues from our July 2008 report. The committee plans to assess DCAA actions on recommendations in these reports and identify any gaps for further action. We note that DCAA has already taken numerous actions to respond to our initial investigative report as well as DOD Comptroller/CFO and DBB recommendations. Long-Term Legislative Actions To Move DCAA: Most of the impairments to DCAA effectiveness that we identified can be addressed within DCAA's current organizational placement. However, to address the Committee's interest in how changes in DCAA's organizational placement could improve DCAA effectiveness and independence, we considered potential approaches to moving DCAA. During the 1980s, there were numerous proposals to reorganize DCAA's organizational structure, including legislative proposals that would have placed DCAA in the Office of the Under Secretary of Defense (Acquisition), or in the DOD Office of Inspector General (OIG), or placed only DCAA's post-contract audits in the OIG. We analyzed these proposals in an April 1991 report[Footnote 101] and concluded that they were not workable because they posed conflict of interest or duplication of effort issues. We believe that it is prudent to consider changes in organizational placement after DCAA has had sufficient opportunity to effectively implement current reform efforts necessary to address fundamental operational issues. Legislation to move DCAA as an organization would require careful analysis and planning before implementation. Moving DCAA at this time would be a bold step with possible unintended consequences, and decision makers would need to carefully weigh the costs and benefits of moving DCAA before the fundamental operational issues are addressed. As discussed below, regardless of its ultimate placement in the government, DCAA still needs to address the fundamental weaknesses in its mission, strategic plan, metrics, audit approach, and human capital management. Elevating DCAA within DOD: Elevating DCAA within DOD as a separate component reporting to the Deputy Secretary of Defense could give more authority to the DCAA Director and increase visibility of the organization both within and outside of DOD.[Footnote 102] Because DOD positions reporting to the Secretary level are established by law, moving DCAA to the department level would require new legislation. To avoid any ambiguities or questions about whether the Secretary of Defense currently possesses the statutory authority to transfer the supervision of DCAA to the Deputy Secretary, we believe additional legislation that sets out appropriate relationships would be the best approach.[Footnote 103] In addition, this option would require some level of administrative change. For example, management and oversight of the contract audit function would become the responsibility of Deputy Secretary, a separate appropriation would need to be established, and some form of periodic external reporting to Congress would be appropriate. We note that authorizing legislation to move DCAA could also include similar protections and authorities as those under the IG Act if these provisions have not already been enacted. Although this option could enhance DCAA auditor objectivity and independence, under this organizational placement, DCAA would still need to resolve the management environment and cultural problems that have had a negative impact on audit quality, including pressure by contractors and contracting officers on audit scope and findings, conclusions, and recommendations. DCAA also would need DOD commitment to strengthening DCAA's contract audit function through continued monitoring and oversight. Leadership from the Deputy Secretary of Defense would be critical to help DCAA address these matters. A key factor will be whether the Deputy Secretary has the necessary time to focus on DCAA. The amount of time needed should be less once the fundamental improvements are accomplished. Establishing an Independent, Governmentwide Contract Audit Agency: Numerous governmentwide acquisition management reform efforts are currently under way that could impact the contract audit function. These efforts include congressional oversight and reform legislation and Presidential direction on developing governmentwide guidance for reviews of existing contracts to identify contracts that are wasteful, inefficient, or otherwise unlikely to meet agencies' needs, and to formulate corrective action in a timely manner, as well as interest group studies. For example, in the National Defense Authorization Act for Fiscal Year 2008, Congress created the Commission on Wartime Contracting to study federal agency contracting for the reconstruction, logistical support of coalition forces, and the performance of security functions in Iraq and Afghanistan. The Senate Committee on Homeland Security and Governmental Affairs also recently created a new Subcommittee on Contracting Oversight. Several Members of the House Oversight and Government Reform Committee created the Clean Contracting Coalition to take a similar governmentwide approach. The House Oversight and Government Reform Committee also has been very active in this area. In addition, the House Armed Services Committee established an acquisition panel to evaluate DOD's current acquisition system, analyze the root causes of project or program failures, and the administrative and cultural pressures that acquisition and program personnel face. The House and Senate Armed Services Committees also led the effort to enact the Weapon Systems Acquisition Reform Act of 2009, [Footnote 104] which requires oversight of cost estimation, systems engineering, and performance assessment; promotes competition; and limits organizational conflicts of interest. On March 4, 2009, the President issued a memorandum directing executive agencies to (1) increase the use of fixed-price contracts, (2) enhance the capacity of the acquisition workforce, (3) maximize competition, and (4) rationalize the choice of government or contractor resources to perform required services. In addition, the Federal Acquisition Innovation and Reform Institute--a nonpartisan, nonprofit organization led by leaders in acquisition and supply management--has called for acquisition workforce reforms, including a single acquisition job series that encompasses at a minimum, three functions--program management, contracting, and a new function called requirements management--and is considered a professional "super COTR" (contracting officer's technical representative) position. Over the next several years, these reform initiatives likely will have a significant impact on government contracting, including the roles and relationships of contract auditors and the contracting, program, and finance communities. Depending on the outcome of the various contract reform initiatives and the successful implementation of DCAA management reforms, Congress may also want to consider increasing the efficacy of these reforms by establishing an independent governmentwide contract audit agency. The creation of a statutory governmentwide contract audit agency could enhance contract auditor effectiveness and independence by placing the audit agency outside DOD and other federal agencies that make procurement and contract management decisions. Centralizing the contract audit function and mandating its use by all federal agencies also could provide for consistent audit coverage and bring efficiencies and economies of scale to the contract audit process across the government. However, this would likely entail significant costs and operational and accountability considerations and would be an extremely costly option involving significant infrastructure and reorganization and would require substantial planning and analysis before deciding whether to proceed and how to implement any changes. Some of the issues that would need further study and analysis include the following: Governance. Governance is the framework of rules and practices by which a governing body, such as a board of directors, ensures accountability, fairness, and transparency in the entity's relationship with all of its stakeholders, including management, employees, and government. In order to improve governance and accountability at federal agencies, a variety of laws covering a range of management and administrative practices and processes have been enacted. Consideration of such provisions for a governmentwide contract audit agency should include application of general laws related to funds control, performance and financial reporting, accounting and internal control systems, human resources management, and recordkeeping and access to information, among others. Further, governance issues unique to a contract audit agency, such as its relationships to agency contracting officers and the Congress, should be assessed. Scope of Work. Scope of work considerations would include roles, responsibilities, and relationships of the governmentwide contract audit agency and IGs with regard to contract audits. Another consideration would be whether the new agency would be available for consultation as an outside expert on federal agency pre-award issues. In addition, a determination would need to be made on the handling of fraud referrals. For example, the central new agency could have an investigative division or it could refer potential contract fraud to federal agency IGs for further investigation. Funding. Congress would need to determine how to fund the new contract audit agency. For example, funding could be provided through appropriations or from reimbursement by federal agencies. This decision would likely be tied to decisions on the governmentwide contract audit agency's mandate and scope of work and any realignment of contract audit resources. Further study and analysis of this option would involve input from the federal agency IGs and agency contracting and finance communities as well as government contractors and public interest groups. Numerous additional issues would potentially be identified and require substantial time and cost for effective consideration and resolution. Conclusions: Successful accomplishment of DCAA reforms will require focused and committed leadership at the highest levels of DOD and DCAA as well as fundamental changes in DCAA's culture and possible congressional action. Without leadership commitment to a strong contract audit function and substantial changes to DCAA's mission, strategic plan, and management environment and culture, DCAA will continue to be challenged in its ability to perform quality audits that protect the public interest. Many needed changes are planned or under way and can be completed in the short-term, including revising DCAA's mission statement, strategic plan, and monitoring, and adjusting performance metrics. Fundamental structural and cultural changes related to developing and implementing a comprehensive, risk-based approach for contract audits that comply with professional auditing standards and identifying staffing, training, and resource needs will take several years to accomplish and implement. However, unless the overall problems with DCAA's culture and management environment that resulted in pervasive contract audit failures are resolved, billions of taxpayer dollars will continue to be at risk for fraud, waste, abuse, and mismanagement. Recommendations for Executive Action: We are making 15 recommendations to the Secretary of Defense to improve the quality of the agency's audits and strengthen auditor integrity, objectivity, and independence, including recommendations for actions on findings in this report that are aligned with certain Defense Business Board (DBB) findings and recommendations. First, we recommend that the Secretary of Defense revise DCAA's mission statement to reflect the need for quality contract audits and related nonaudit services that take into account serving the public interest. We also recommend that the Secretary of Defense require the Under Secretary of Defense (Comptroller/CFO) to establish milestones for completing DCAA corrective actions and monitor and regularly report on DCAA progress to assure timely completion of critical actions. In addition, we recommend that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the Defense Contract Audit Agency (DCAA) to take the following 13 actions. The following five recommendations cover actions to address our findings that are similar to DOD Comptroller/CFO and DBB findings. * In concert with the revised mission statement, develop a strategic plan with short-term and long-term outcome-related goals. * To measure progress in achieving strategic goals, ensure that metrics are tied to the revised mission statement and strategic plan and support the agency's annual work plan. * Consult with DOD stakeholders and engage outside experts to develop a risk-based contract audit approach that identifies resource requirements and focuses on performing quality audits that meet generally accepted government auditing standards (GAGAS). * Establish an SES-level position with responsibility for audit quality assurance that requires demonstrated knowledge and experience in applying professional audit standards. * Consistent with DBB report observations, establish a separate DCAA internal review organization to conduct critical internal inspector general functions, including performing periodic internal evaluations and reviews and addressing DCAA hotline complaints. The following eight recommendations relate to specific GAO findings in this report. * In consultation with DOD stakeholders, review DCAA's current portfolio of audit and nonaudit services to determine if any should be transferred or reassigned to another DOD agency or terminated in order for DCAA to comply with GAGAS integrity, objectivity, and independence requirements. * Based on the risk-based audit approach, develop a staffing plan that identifies auditor resource requirements as well as auditor skill levels and training needs. * Establish a position for an expert on auditing standards or consult with an outside expert on auditing standards to assist in revising contract audit policy, providing guidance on sampling and testing, and developing training on professional auditing standards. * Revise DCAA audit policy to provide appropriate guidance on what constitutes sufficient testing to comply with GAGAS. Update DCAA's Contract Audit Manual, as appropriate. * Develop agencywide training on government audit standards. This training should emphasize the level of assurance intended by the various types of engagements and provide detailed guidance on auditor independence, planning, fraud risk, level of testing, supervision, auditor judgment, audit documentation, and reporting. * Conduct a comprehensive, independent review of DCAA's revised audit quality assurance function. This review should focus on the consistent application of criteria used for assessing audit quality and assuring timely, consistent, and appropriate reporting of review results. * Make appropriate recommendations to address annual quality assurance review findings of serious deficiencies and GAGAS noncompliance, provide training, and follow-up to assure that appropriate corrective actions have been taken. * Establish policies and procedures to ensure that auditors who make direct bill decisions are independent of DCAA employees who perform a DOD management function by reviewing vouchers of contractors not eligible for the direct billing program, thereby reducing situations where DCAA auditors are encouraged to reduce their office workload by approving contractors for the direct-bill program. Further, we recommend that the Department of Defense Inspector General take the following two actions. * Reconsider its overall conclusions in the May 2007 DOD IG report on the audit of DCAA's quality control system in which it reported an adequate ("clean") opinion on DCAA system of quality control in light of the serious deficiencies and findings included in that report and the additional evidence identified in our audit. * Based on the above, determine whether the report should be rescinded or modified. Matters for Congressional Consideration: In addition to our recommendations to DOD for improving DCAA audit quality and auditor objectivity, integrity, and independence, Congress may wish to consider the following legislative actions for enhancing DCAA's effectiveness and independence. In considering these options, the Congress would need to weigh DCAA's ability to accomplish significant reforms within its current environment and the cost and administrative effort involved with the alternative options along with the potential benefits. Timing would also need to be considered, given significant reforms that DCAA is already undertaking and the additional burden that a change in organizational placement would add at this time. * In the short term, as DCAA makes progress in correcting fundamental weaknesses that have impacted audit quality, Congress could consider enhancing DCAA reform efforts by enacting legislation to grant it protections and authorities similar to those embodied in the Inspector General Act, as amended. * In the medium term, Congress could consider elevating the contract audit function within DOD by moving DCAA from under the DOD Comptroller/CFO and placing it under the Deputy Secretary of Defense. * In the longer term, depending on the outcome of acquisition management reform initiatives under way and the success of DCAA management reforms, Congress could consider creating an independent, governmentwide contract audit agency. Legislation to move DCAA should incorporate the protections and authorities similar to those embodied in the Inspector General Act, if these have not already been granted to DCAA. Agency Comments and Our Evaluation: We made a total of 17 recommendations, including 15 recommendations to DOD to improve DCAA's management environment, audit quality, and oversight; and we made 2 recommendations to the DOD IG regarding DCAA's last peer review. We received written comments from the Department of Defense (DOD) on September 8, 2009, and we received written comments from the DOD Inspector General (IG) on September 3, 2009. DOD stated that the department concurs with all but one of our 15 recommendations. DOD also stated that the Department and DCAA are committed to taking the necessary corrective actions to address our findings and that the department will continue to monitor DCAA to ensure timely completion of critical actions to address our recommendations. DOD also provided comments on our matters for congressional consideration. Although DOD disagreed with the matters we discussed, we continue to believe these are valid matters for congressional consideration. The DOD IG concurred with our recommendation to reconsider the conclusions in its May 2007 peer review report on DCAA; the IG did not concur with our recommendation to determine whether to rescind or modify its peer review report. DOD's written comments are reprinted in appendix IV, and the DOD IG's written comments are reprinted in appendix V. We summarize and evaluate the DOD and DOD IG comments and responses to our recommendations below. We made technical corrections and clarifications suggested by DOD in the body of our report, where appropriate. DOD Comments and Our Response: DOD's written comments include (1) comments on our 15 recommendations, (2) comments on matters we presented for congressional consideration, (3) a list of DCAA corrective actions, (4) DCAA clarifications, and (5) comments from the Director, Defense Procurement and Acquisition Policy. DOD officials fully concurred with 13 of our 15 recommendations for improving DCAA audits, partially concurred on one recommendation, and did not concur with one recommendation. We view DOD comments as being generally responsive to the intent of our recommendations. Our discussion of DOD's response to our matters discussion and our findings and recommendations follow. We provide additional comments on specific sections of the DOD response letter in appendix IV. With regard to the matters we presented for congressional consideration, DOD stated that it generally opposes providing DCAA with authorities similar to those contained in the Inspector General Act. DOD stated that it specifically opposes certain recommendations based on the IG model if DCAA remains within DOD, including (1) a Presidentially-appointed and Senate-confirmed DCAA Director, unless DCAA is independent of DOD, (2) fixed terms for the DCAA Director, (3) an independent budget, and (4) mandatory public reporting. DOD also stated that it plans to take steps to strengthen DCAA's independence by establishing an appeals process that permits DCAA to seek resolution when there are differences of opinion as to the resolution of its audit findings. Finally, DOD opposes moving DCAA from under the DOD Comptroller/CFO and placing it under the Deputy Secretary. DOD pointed out that the Deputy Secretary is the Chief Management Officer of one of the world's largest organizations and backs up the Secretary in the wartime chain of command, and he does not have the time to provide oversight and support to individual defense agencies. Although DOD did not agree with these matters, we believe they provide important information for Congress to consider. For example, the Inspector General Act provides many important authorities and protections for IG's that could enhance DCAA's independence and effectiveness. DOD disagreed with the Presidential appointment and Senate confirmation provision because it believes this would inject a political element into DCAA that is not appropriate and could create lengthy periods where there is no Director. DOD also opposes fixed terms for the DCAA Director because it believes the Secretary of Defense must have the ability to choose an appropriate Director. Our position with regard to appointments of IGs has been that Presidential appointments with Senate confirmation enhance their independence from the entities they audit and investigate. We recognize that DCAA serves a different role than IGs. We looked to the IG Act model to identify provisions that enhance the independence of auditors. A political appointment would elevate the status of the Director among DCAA's stakeholders and, as a consequence, give DCAA more authority to respond to actions taken by its stakeholders to influence its independent audit work. A fixed term would provide stability, especially during a time of organizational change. DOD also questioned the wisdom of an independent budget because it would limit its ability to move money into DCAA, as is occurring now based on funding from the Defense Acquisition Workforce Development Fund. Separate appropriations are a key independence provision for IGs. The ability to reprogram funds within the Defense-wide Operations and Maintenance appropriation can involve both increases and decreases. Our analysis of DCAA reprogrammings over the last three years showed that funds were also moved from DCAA to other DOD organizations within the Defense-wide operations and maintenance appropriation. In DCAA's case, the reprogrammings to reduce funding generally related to large unobligated balances--showing that DCAA under executed its budget. We believe this is important information that Congress would want to know. Further, we do not see a reason why DOD could not receive approval to transfer funds to DCAA from another fund if it had a separate budget. For example, providing DCAA with funds from the Defense Acquisition Workforce Development Fund constitutes a transfer (not a reprogramming), the authority for which is provided in the legislation governing the Fund, 10 U.S.C. 1705(e). DOD also opposed mandatory public reporting by DCAA. We believe that periodic reporting to Congress and the public on the results of DCAA's work will enhance accountability over DCAA. As discussed in our report, DCAA needs time to address the fundamental weaknesses in mission and the overall management environment. However, if DCAA is not successful in resolving these problems under its current organizational placement, it will be necessary to consider additional actions. In this regard, it may be worthwhile to consider elevating DCAA as a component agency reporting to the Deputy Secretary because this could enhance DCAA's independence by providing it more authority within DOD and increase DCAA's visibility both within and outside of DOD. DOD partially concurred with our recommendation that DCAA consult with DOD stakeholders and engage outside experts to develop a risk-based contract audit approach that identifies resource requirements and focuses on performing quality audits that meet GAGAS. DOD stated that DCAA already has a risk-based contract audit approach in identifying resource requirements and considers audit risk in planning various assignments. DOD stated that DCAA will coordinate with the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD (AT&L)) to assess DCAA audit requirements.[Footnote 105] DOD also noted that one of DCAA's cultural transformation projects is identifying and resolving differing stakeholder expectations while ensuring DCAA performs quality audits that meet GAGAS. DOD expects to complete its assessment of stakeholder needs based on regulatory and statutory requirements by December 2010. We appreciate these steps; however, we remain concerned that DCAA's current approach of performing 30,000 to 35,000 audits and issuing over 22,000 audit reports with 3,600 auditors substantially contributed to the widespread audit quality problems we identified. Generating that many reports and doing that many audits with 3,600 auditors leaves very little time to perform in-depth, complex audits of contractors. While the Director of Defense Procurement and Acquisition Policy commented that contract audits need to be completed "in time to be useful," to assure timely, quality audits, DCAA will need a risk-based approach to determine the appropriate level of audit and nonaudit effort and staffing. DOD did not concur on our recommendation to develop policies and procedures related to direct-billing decisions, stating that (1) the department believes that a review of the contractor's interim public vouchers is an integral function of DCAA's continual assessment of a contractor's billing system (2) DCAA is in the best position to review and approve contract interim billings based on its thorough understanding of the contractor's system, (3) DOD believes that our concerns are mitigated based on the comprehensive supervisory and audit manager reviews, and (4) DCAA does not believe that the approval of interim vouchers along with the approval for contractors to be on direct billing results in a lack of auditor objectivity. We continue to believe that DCAA's management (nonaudit) responsibility to perform prepayment reviews of contractor vouchers for DOD and the auditor's decision making role of approving contractors for direct billing privileges based on its audit conclusions about the strength of the contractor's system of internal controls, create audit objectivity issues. We revised our findings discussion and our recommendation to clarify this point. Under normal circumstances, DCAA must review contractor vouchers prior to payment--a management support function for DOD generally performed by DCAA field office administrative staff. By obtaining direct billing privileges, contractors can receive payment for goods and services without a voucher review by DCAA prior to payment. Because we found that this situation provides an incentive for DCAA to reduce its administrative workload by recommending that contractors are placed on direct billing, we recommended that DCAA develop new policies and procedures to ensure a separation between staff reviewing vouchers and staff making direct-bill decisions. In addition, DCAA has not explained the basis for its belief that administrative staff have a thorough understanding of the contractors' systems. Further, we disagree with DOD's statement that our concerns are mitigated based on the comprehensive supervisory and audit manager reviews because this is not supported by our findings. The fact that DCAA approvals of contractor direct-bill privileges were not based on sufficient audit procedures as demonstrated by our work and DCAA's removal of over 200 contractors from the direct-bill program since our July 2008 report[Footnote 106] support our concern that the existence of such an incentive presents an objectivity impairment. DOD provided additional comments on findings in its transmittal letter. DOD stated that it disagrees with the suggestion in our report that the department has not yet begun to address the weaknesses we identified. Our report neither states nor implies that DOD has not yet begun to take action. In fact, one of our objectives was to analyze steps DOD has taken so far, and our report describes in detail the progress made. Our report acknowledges that several positive steps have been taken by DCAA, but much more needs to be done to address the fundamental problems. Thus, solutions to the problems documented in this report will take time to first implement and then will have to be independently assessed to make sure they are effective. Our report also notes that fundamental changes have not taken place. For example, to date DOD has not revised DCAA's mission statement to reflect the need to consider the public interest as a key component of its work. In addition, DCAA has yet to assess the feasibility of 3,600 auditors issuing over 20,000 reports in one year (22,349 in fiscal year 2008) and the appropriateness and need for the current combination of audit and non-audit services that drives this workload. Until these and other key steps are further along, it will be too early to assess whether DCAA has fundamentally changed or whether past practices continue. The DOD comments noted that one of our major findings is the lack of sufficient testing to support conclusions when giving an opinion on contractor internal control systems. The comments incorrectly refer to the requirement for sufficient testing as a GAO requirement and state that planned staffing increases may not be enough to accomplish audits required by regulation in light of additional testing stipulated by GAO. Professional audit standards have always required auditors to obtain sufficient evidence to provide a reasonable basis for the conclusion expressed in the report. As stated in our report, testing methodologies are a matter of professional judgment and can involve many factors. However, our findings reflect more than a difference of opinion with DCAA auditors on their exercise of professional judgment as reflected, in part, by the number of audit reports DCAA rescinded for insufficient testing. For example, we found insufficient documentation to support the methodology chosen and insufficient reasons for minimal testing, such as being told by a DCAA auditor that a "file size was too large" to test more than two recent vouchers. Again, DOD must address the feasibility of 3,600 auditors issuing over 22,000 reports annually, most of which were reportedly performed under auditing standards. This may entail not only a risk-based audit approach but also exploring changes to the regulations that DOD represents require tens of thousands of these audits. DOD also disagreed with our position on the status of actions to strengthen DCAA's quality assurance program. DOD stated that DCAA has been proactive in standing up its new Integrity and Quality Assurance Directorate. DOD also stated that it believes the extensive overhaul of the quality assurance function accomplished in fiscal year 2009 will mitigate the prior shortcomings in audit quality that we cited. Although DOD's comments imply that DCAA has resolved its quality assurance problems, DCAA has acknowledged that it is not ready to undergo another peer review at this time. On September 1, 2009, we received a letter from the DCAA Director, stating that although improvements were put in place in fiscal year 2009, several significant improvements will be accomplished in fiscal year 2010. To allow sufficient time for DCAA to fully implement the necessary corrective actions, DCAA contacted us for guidance on (1) deferring its external quality control review for 2 years and (2) requesting that the next external peer review to cover assignments to be completed in fiscal year 2011. We agree with DCAA that it is not cost-effective to undergo an external peer review until an adequate system of quality control is in place. Expending substantial DOD IG resources when DCAA acknowledges that several years are necessary for improvements to be fully implemented is, in our view, an inefficient use of resources. DCAA has already begun to appropriately disclose in its reports that its audits do not comply with GAGAS external peer review requirements. The Director of Defense Procurement and Acquisition Policy (DPAP) provided additional comments. The Director stated that our report impugns DCAA's audits and that we adopt the position that because DCAA is serving the interests of contracting officers, DCAA is therefore not auditing in the interest of the public. The Director further asserts that DCAA serves the public interest by providing useful and timely information to contacting officers, and that it is erroneous to imply that contracting officers do not seek to protect the public interest. Also, the Director states that GAO agrees with the Defense Business Board's recommendation to revise DCAA's mission to reflect a focus on the taxpayer as the primary customer. Finally, the Director suggests that our criticism of DCAA's "production-oriented auditing" sets up a dichotomy between quality and timely audits. We disagree with these characterizations of our report. DPAP's statements that our report impugns DCAA's audits and that we take the position that when DCAA is serving the interests of contracting officers, it is not auditing in the interest of the public relate to our summarization of the Defense Business Board (DBB) report and not our findings. Our report does not endorse the specific recommendations of the DBB to focus on the taxpayer as the primary customer. As our report points out, this recommendation does not take into account the regulatory and policy requirements that establish DCAA's primary role as an advisor to government contracting officers and disbursing officers. However, we agree with the DBB that DCAA should consider the public interest when carrying out GAGAS engagements. For audits and attestation engagements conducted under GAGAS, the auditor is expected to objectively and independently acquire and evaluate sufficient, appropriate evidence, and report on the results, consistent with the guidance in GAGAS.[Footnote 107] GAGAS states the principle that "observing integrity, objectivity, and independence in discharging [auditors'] professional responsibilities assists auditors in meeting the principle of serving the public interest and honoring the public trust."[Footnote 108] DPAP also stated that the contracting officer is bound by regulation to meet the public interest in the broadest sense, for the entire matter surrounding a contract and that this includes factors other than DCAA audit findings and recommendations. As reflected in the extensive background discussion and elsewhere throughout our report, we recognize that contracting officers make final contracting decisions, and DCAA engagements support contracting officers in that process. Because we did not review the standards that contracting officers must follow, we did not include references to the requirements for contracting officers to protect the public interest in their actions. Our report also states that DCAA contract audit services are intended to be a key control to help assure that prices paid by the government for needed goods and services are fair and reasonable and that contractors are charging the government in accordance with applicable laws, regulations (e.g., Federal Acquisition Regulation (FAR) and Defense Federal Acquisition Supplement (DFARS), standards (e.g., Cost Accounting Standards (CAS)), and contract terms. In providing this assurance, DCAA audits would necessarily take into account serving the public interest. However, when DCAA audits do not meet GAGAS, they do not provide this assurance and thus are not serving the public interest. We found that DCAA auditors lacked objectivity and independence when performing GAGAS audits and engagements. In many cases, this was a result of auditors' focus on expediency to support client needs and, as the Director also observes, human capital shortages and poor management decisions. We do not question the need for contracting officers to use the services of advocates and assistants in carrying out their duties. However, when contract auditors represent that they are performing engagements under GAGAS, their primary focus should be on the integrity, objectivity, and independence of their work, which serves both contracting officers and the public interest. Further, the quality of DCAA audits impacts the quality of information available for contracting officer decisions. Whether DCAA can adhere to GAGAS on contract audits that provide minimal time to perform the work is a factor that USD (AT&L) should consider when establishing requirements for contract audit services. As we recommended, DOD should reconsider the mix of audit and non-audit services that it needs. DOD IG Comments and Our Response: The DOD IG concurred with our recommendation to reconsider its overall conclusions in the May 2007 report on the audit of DCAA's quality control system in which it reported an adequate ("clean") opinion on DCAA's system of quality control in light of the serious deficiencies and findings included in that report and the additional evidence identified in our audit. The IG also stated that it did not concur with our recommendation to rescind the report and, because of that statement, we believe the IG misconstrued our recommendation as expressly calling for a rescission or modification of its peer review report. Our recommendation was for the IG to determine, based on the results of our recommended reconsideration of the IG's conclusions, whether it should rescind or modify the peer review report. The DOD IG also states that it took alternative action that conformed to the intent of our recommendation. The DOD IG comments state that it notified DCAA on August 24, 2009, that the May 2007 "adequate" opinion on DCAA's system of quality control would expire on August 26, 2009. In addition, the IG stated, "We have determined that it is not prudent to allow the adequate opinion from our May 2007 report to carry forward." However, peer review opinions neither "expire" nor "carry forward" beyond the period covered by the peer review. Peer review opinions cover the period to which the opinion applied--in DCAA's case, as of the end of fiscal year 2006--and the peer reviewed audit organization need not undergo another peer review during the next 2 years.[Footnote 109] Because it has been more than 3 years since DCAA's last peer review, DCAA is no longer in compliance with the GAGAS requirement for an external peer review, and DCAA has taken appropriate action to disclose this noncompliance in its reports. As stated in our report, the overall conclusion in the DOD IG report is inconsistent with the detailed observations in its report, which indicate numerous significant deficiencies in DCAA's system of quality control. Further, based on DCAA's actions to rescind 80 audit reports, 39 of which were issued in fiscal year 2006--the period on which the IG conclusions are based--and the findings in our audit, we concluded that DCAA's quality control system for the period covered by the DOD IG peer review was not effectively designed and implemented to provide assurance that DCAA and its personnel comply with professional standards. As agreed with your office, unless you publicly announce the contents of this report, we plan no further distribution for 30 days from the report date. At that time, we will send copies of this report to the Secretary of Defense; the Under Secretary of Defense (Comptroller/CFO); the Under Secretary of Defense for Acquisition, Technology, and Logistics; the DOD Director for Defense Procurement and Acquisition Policy; the Deputy General Counsel for Acquisition; the Secretary of the Army; the Secretary of the Navy; the Secretary of the Air Force; the Director of DCAA, the Director of DCMA; the DOD Inspector General; and the Director of the Office of Management and Budget. In addition, this report will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. If you or your staff have any questions concerning this report, please contact me at (202) 512-7922 or kutzg@gao.gov or Gayle Fischer, Assistant Director, Financial Management and Assurance at (202) 512- 9577 or fischerg@gao.gov. Signed by: Gregory D. Kutz: Managing Director: Forensic Audits and Special Investigations: [End of section] Appendix I: Internal Control System Audits Did Not Meet Professional Standards: In performing its audits, the Defense Contract Audit Agency (DCAA) states that it follows generally accepted government auditing standards (GAGAS).[Footnote 110] As part of our assessment of DCAA's overall management environment and quality assurance structure, we reviewed documentation for selected DCAA audits of contractor systems controls for compliance with GAGAS. We focused on internal control audits because contracting officers rely on DCAA audit opinions on contractor system controls for 3 or more years to make decisions on pricing and contract awards and DCAA uses the audit opinions to assess risk when planning subsequent audits. We selected seven DCAA field audit offices (FAO) across the five DCAA regions that reported predominately adequate ("clean") opinions on contractor controls. For the seven FAOs, we reviewed 37[Footnote 111] selected audits of contractor internal control systems, including accounting, estimating, billing, and indirect and other direct cost systems. As shown in table 5, we assessed these audits for compliance with eight key areas of GAGAS requirements: (1) auditor independence;[Footnote 112] (2) adequate planning; (3) auditor understanding of controls; (4) design of procedures to detect risk of fraud, abuse, mismanagement, and contract terms; (5) documentation of sampling and testing; (6) audit evidence supports conclusions and opinion; (7) proper supervision; and (8) timely reporting and disclosures.[Footnote 113] We also considered GAGAS requirements for protecting the public interest when using auditor judgment.[Footnote 114] As discussed in the body of this report, the 37 audits we reviewed did not comply with GAGAS in one or more of these areas. However, we determined that 4 of the 37 audits included sufficient testing to support reported conclusions and opinions. Because the conclusions and opinions in the deficient audits were used to make risk assessments and determine the level of testing in other DCAA audits, such as annual audits of contract or incurred cost claims, audits of contract proposals and contractor forward pricing proposals, progress pay audits, and contract close-out audits, the audit quality issues related to the GAGAS noncompliance we identified potentially impacts hundreds of other audits and contracting decisions covering billions of dollars in DOD expenditures. Table 5: GAGAS Noncompliance on 37 Selected Audits of Contractor Controls: Reasons for GAGAS noncompliance: Independence impairments; DCAA regions: Northeast: (FAO #1): 0; DCAA regions: Mid-Atlantic: (FAO #2): 0; DCAA regions: Eastern: (FAO #3): 1; DCAA regions: Central (FAO #4 & 5): 2; DCAA regions: Western (FAOs #6 & 7): 4; DCAA regions: All regions: 7. Reasons for GAGAS noncompliance: Inadequate planning; DCAA regions: Northeast: (FAO #1): 0; DCAA regions: Mid-Atlantic: (FAO #2): 4; DCAA regions: Eastern: (FAO #3): 1; DCAA regions: Central (FAO #4 & 5): 5; DCAA regions: Western (FAOs #6 & 7): 7; DCAA regions: All regions: 17. Reasons for GAGAS noncompliance: Inadequate auditor understanding of controls; DCAA regions: Northeast: (FAO #1): 1; DCAA regions: Mid-Atlantic: (FAO #2): 4; DCAA regions: Eastern: (FAO #3): 0; DCAA regions: Central (FAO #4 & 5): 2; DCAA regions: Western (FAOs #6 & 7): 5; DCAA regions: All regions: 12. Reasons for GAGAS noncompliance: Lack of fraud risk detection procedures; DCAA regions: Northeast: (FAO #1): 4; DCAA regions: Mid-Atlantic: (FAO #2): 5; DCAA regions: Eastern: (FAO #3): 6; DCAA regions: Central (FAO #4 & 5): 11; DCAA regions: Western (FAOs #6 & 7): 9; DCAA regions: All regions: 35. Reasons for GAGAS noncompliance: Insufficient documentation on sampling methodology; DCAA regions: Northeast: (FAO #1): 3; DCAA regions: Mid-Atlantic: (FAO #2): 5; DCAA regions: Eastern: (FAO #3): 6; DCAA regions: Central (FAO #4 & 5): 9; DCAA regions: Western (FAOs #6 & 7): 4; DCAA regions: All regions: 27. Reasons for GAGAS noncompliance: Insufficient evidence to support conclusions and opinion; DCAA regions: Northeast: (FAO #1): 3; DCAA regions: Mid-Atlantic: (FAO #2): 5; DCAA regions: Eastern: (FAO #3): 6; DCAA regions: Central (FAO #4 & 5): 11; DCAA regions: Western (FAOs #6 & 7): 8; DCAA regions: All regions: 33. Reasons for GAGAS noncompliance: Improper Supervision; DCAA regions: Northeast: (FAO #1): 0; DCAA regions: Mid-Atlantic: (FAO #2): 0; DCAA regions: Eastern: (FAO #3): 2; DCAA regions: Central (FAO #4 & 5): 5; DCAA regions: Western (FAOs #6 & 7): 6; DCAA regions: All regions: 13. Reasons for GAGAS noncompliance: Reporting problems; DCAA regions: Northeast: (FAO #1): 4; DCAA regions: Mid-Atlantic: (FAO #2): 5; DCAA regions: Eastern: (FAO #3): 6; DCAA regions: Central (FAO #4 & 5): 12; DCAA regions: Western (FAOs #6 & 7): 10; DCAA regions: All regions: 37. Total GAGAS noncompliance issues: DCAA regions: Northeast: (FAO #1): 15; DCAA regions: Mid-Atlantic: (FAO #2): 28; DCAA regions: Eastern: (FAO #3): 28; DCAA regions: Central (FAO #4 & 5): 57; DCAA regions: Western (FAOs #6 & 7): 53; DCAA regions: All regions: 181. Number of audits: DCAA regions: Northeast: (FAO #1): 4; DCAA regions: Mid-Atlantic: (FAO #2): 5; DCAA regions: Eastern: (FAO #3): 6; DCAA regions: Central (FAO #4 & 5): 12; DCAA regions: Western (FAOs #6 & 7): 10; DCAA regions: All regions: 37. Rescinded reports: DCAA regions: Northeast: (FAO #1): 1; DCAA regions: Mid-Atlantic: (FAO #2): 3; Eastern: (FAO #3): 3; DCAA regions: Central (FAO #4 & 5): 7; DCAA regions: Western (FAOs #6 & 7): 4; DCAA regions: All regions: 18. Source: GAO analysis of selected DCAA audits. Note: Because of the large size of the Central and Western regions, we tested audits at more than one field audit office in these regions. [End of table] The following discussion includes examples of GAGAS noncompliance from specific audits we reviewed. Independence Impairments: GAGAS state that the audit organization and the individual auditor should be free, both in fact and appearance, from personal, external, and organizational impairments to independence.[Footnote 115] Our review of 37 audits of contractor internal controls found evidence in documentation for 7 audits that DCAA independence was compromised because auditors provided material nonaudit services to a contractor they later audited; experienced access-to-records problems that were not resolved; delayed report issuance, which allowed the contractor to resolve cited deficiencies, without proper reporting; and performed test work on billings the contractor selected for testing. GAGAS state that auditors should be free from influences that restrict access to records or improperly modify audit scope.[Footnote 116] GAGAS also state that audit organizations should not audit their own work or provide nonaudit services if the services are significant or material to the subject matter of the audit.[Footnote 117] The following examples describe a situation where auditors assisted a Department of Defense (DOD) contractor in developing billing system policies and procedures after identifying five significant deficiencies and then reviewed their own work during a follow-up audit. Text box: DCAA Auditors Issued an Adequate Opinion on Controls They Helped Design; DCAA auditors impaired their independence by performing nonaudit services for one of the top five DOD contractors in terms of dollars when they helped the contractor develop policies and procedures that were material to the billing system they were auditing. On May 12, 2005, DCAA reported an inadequate-in-part opinion on the contractor's billing system internal controls. The report included five significant deficiencies, including a failure to maintain current, adequate billing system policies and procedures. After issuing the report, DCAA auditors helped the contractor develop adequate policies and procedures related to accounts receivable, overpayments, and monitoring of the billing system before performing the required follow- up audit--an impairment to auditor independence. A year later, after performing the follow-up audit, DCAA auditors concluded that the contractor had performed adequate actions to correct all of the billing system deficiencies previously reported. On June 28, 2006, DCAA reported an adequate opinion on the contractor's billing system internal controls. Following GAO's review of these audits, on March 6, 2009, DCAA rescinded the billing system audit follow-up report. [End of text box] We also noted instances of denials and limitations on access to records by contractors that were not handled properly. For example, during a billing system audit of one of the top five DOD contractors, an e-mail message documented in the audit workpapers showed that the auditors were challenged by a contractor official when they requested documentation to test whether billing clerks had received required training. The contractor's e-mail stated, "Here's a question for you. Can you tell me who and what requirement is making this part of the [audit]. This is a question that [is] being asked by [the Cash Manager]." The auditors eventually obtained limited training documentation from the contractor. Audit documentation and our interviews with the auditors revealed that the auditors also limited testing in several other areas "because the contractor would not appreciate it." Access to records problems and strong external influence to limit testing are both impairments to independence according to GAGAS.[Footnote 118] Inadequate Planning: GAGAS for attestation engagements state that the work shall be adequately planned.[Footnote 119] Auditors should communicate information regarding the nature, timing, and extent of planned testing and reporting to officials of the audited entity and to the individuals contracting for or requesting the attestation engagement. Auditors should also plan work to follow up on actions to address significant findings and recommendations in previous audits and assess areas of risk in planning the engagement. However, our review of the audit documentation determined that 17 of the 37 internal control audits we reviewed were not adequately planned. * In five audits, auditors failed to consider risk associated with new systems that had not yet been audited or systems that had not been audited in more than 4 years. These audits should have been ascribed a higher risk level according to DCAA CAM guidance and testing should have been increased.[Footnote 120] DCAA has rescinded three of the five audits and is planning new audits, as appropriate. * Without documenting the basis for their decisions, auditors deleted audit steps from standard audit programs or did not perform all audit steps for three of seven audits we reviewed at one FAO. When we asked the auditors why they omitted key audit procedures in their work, the auditors told us they used "auditor judgment." However, the auditors would not explain the basis for their judgments to us or their rationale for omitting key procedures, such as assessing the contractor's control environment and testing the implementation of a contractor's policies and procedures. Because we did not find any justification for omitting key audit procedures in these three audits, we determined that they were inadequately planned. The following case discussion illustrates deficiencies in audit planning as well as a lack of auditor understanding of contractor processes and controls. Text box: DCAA Erroneously Performed an Audit over a Billing System That Did Not Exist; In 2004, a Western Region FAO planned a billing system audit of a federally funded research and development center (grantee) that receives $1.5 billion annually for research services. However, the planning for this billing system audit did not take into account the fact that grantees are funded through letters of credit and do not actually bill the government. This financial relationship is very different--and much less complicated--than a situation where a contractor bills the government for contract costs in accordance with Cost Accounting Standards and the Federal Acquisition Regulation. For example, under a letter of credit financing arrangement, grantees draw funds as disbursements are made and are required to prepare reports of transactions on their use of the funds and submit them to the funding agency. Despite this obvious mistake, on May 6, 2005, DCAA auditors issued a report stating that the grantee had an "adequate billing system."; Another report issued by the same DCAA office on June 25, 2004, reviewed the grantee's cash management practices under the Single Audit Act for another federal agency. The auditors could have simply forwarded this report to the DOD contracting officer--a task that would take an hour at the most to complete. Instead, DCAA auditors charged over 530 staff hours to generate documentation to meet DCAA's billing system audit requirements, even though there was no related "billing system." As a result of our review, DCAA reassessed the need to perform a billing system audit for the grantee and determined that it would rely on the Single Audit Act reports in the future. DCAA has not rescinded the audit report even though it expresses an opinion on a nonexistent system. [End of text box] Auditors Did Not Properly Document Understanding of Controls for Several Audits: GAGAS require that in planning examination-level attestation engagements, auditors should obtain a sufficient understanding of internal control that is material to the subject matter and design procedures to achieve the objectives of the audit.[Footnote 121] The subject matter or assertion the auditor is testing may relate to the effectiveness and efficiency of operations, including the use of an entity's resources; the reliability of financial reporting, including reports on budget execution and other reports for internal and external use; compliance with applicable laws and regulations, provisions of contract, or grant agreements; and safeguarding of assets.[Footnote 122] Although most of the 37 internal control audits we reviewed met this standard, 12 audits did not. The following case study shows an example of insufficient understanding of controls. * On five audits, auditors overstated the strength of the contractor's control environment in the audit documentation and used this information to justify performing little or no testing of controls for accounting and billing system control audits. On two of the five audits, a Mid-Atlantic Region auditor admitted that he included inaccurate statements in the audit documentation. These statements indicated that the contractor performed internal audits and had a formal management-level monitoring process over accounting and billing functions. When we requested copies of the internal audits and management reviews, the auditor admitted that these statements were not true and that he had made "mistakes." He entered the factually incorrect information in the audit documentation to justify performing little or no testing. The auditor was a GS-13 technical specialist who reviewed the work of other auditors and provided them audit guidance. DCAA has rescinded four of the five audits and is planning or initiating new audits. * On another audit involving a business segment of a third contractor of the top 5 DOD contractors, auditors did not consider the contractor's control environment in planning an audit of a new accounting system--a significant factor that resulted in insufficient testing. Further, after identifying significant accounting system deficiencies, including that certain contract costs are manually processed, are not processed timely, or are not adequately reconciled to actual incurred costs, the auditors delayed issuance of the audit report for about 16 months, waiting to see if the contractor would take corrective actions on the identified deficiencies. Although test procedures were applied from February 25, 2004, to September 15, 2004, DCAA reported an "inadequate-in-part" opinion on the contractor's accounting system on March 14, 2006--nearly 1-1/2 years later, without performing any additional testing. Following discussions with GAO, DCAA rescinded this audit report on November 20, 2008. In the audit described below, the auditor relied on the contractor to document the auditor's understanding of controls. [Text box: DCAA Auditors Relied on the Contractor to Document Internal Controls without Testing the Accuracy of the Documentation; This case involves a billing system audit DCAA conducted in 2006. The last time DCAA had tested the billing system for this contractor was in 2000--a clear indication that new tests should be performed. At that time, DCAA's CAM required that contractor internal control systems be audited every 2 to 4 years. However, rather than re-testing the billing system, DCAA auditors provided the contractor's Information Systems Manager with 6- year-old documentation obtained during a DCAA auditor's walkthrough of the billing process in the prior audit. The DCAA auditors asked the Information Systems Manager to update the documentation by making edits where necessary. According to the audit workpapers, the 6-year-old documentation was "edited by the contractor" and provided back to the DCAA auditor. Based on the contractor's documentation of the billing system internal controls, the auditor concluded "we can limit our testing of management reviews, policies and procedures, and implementation of policies and procedures." The auditor then traced one paid voucher through the billing process. This procedure relates to determining whether the auditor's understanding of the process is correct and is not substantive testing (i.e., detailed tests of transactions and balances and analytical review procedures.) The auditor told GAO that she used this "low-risk approach" because she felt that the contractor's system was "strong" and did not warrant a higher risk approach. However, according to the documentation GAO reviewed, the billing system was a software package that downloads accounting system data to spreadsheets. Manual calculations were then used to develop invoice amounts--a process that is prone to errors and does not provide assurance of consistent systematic processing of invoices. Further, since DCAA's earlier walkthrough in 2000, the contractor had experienced significant downsizing and restructuring. The auditor performed no testing of the contractor's billing system controls in order to determine whether the system was operating effectively at the time of the audit. The audit report that was issued on June 21, 2006, with an adequate opinion was not based on sufficient audit procedures to provide assurance over approximately $76 million in sales to the government. After GAO raised concerns about this audit, DCAA rescinded the audit report on March 3, 2009. [End of text box] Failure to Design and Perform Procedures to Detect Fraud Risk: For DCAA examination-level attestation audits of contractor controls that we reviewed, GAGAS requires auditors to design and perform audit steps to obtain reasonable assurance of detecting fraud, illegal acts, or violations of provisions of contracts that could have a material effect on the subject matter of the engagement or internal control. [Footnote 123] DCAA management asserts that its examination-level audits are designed to provide this assurance, and DCAA internal guidance requires auditors to consider a list of fraud indicators included in DCAA's CAM[Footnote 124] or the DOD Inspector General's Handbook on Fraud Indicators[Footnote 125] in planning and performing their work. However, for 35 of the 37 internal control audits we reviewed there was no evidence that DCAA auditors designed specific procedures to identify risk of fraud, illegal acts, violations of contract terms, or other improprieties. Further, our analysis of audit workpapers showed that DCAA auditors lacked an understanding of fraud indicators associated with weak internal controls. For example, although segregation of duties is a key fraud-prevention control, in the seven audits where workpapers identified segregation of duties issues, the auditors did not consider a lack of segregation of duties to be a fraud risk in 6 of the audits. The auditors did not look for a compensating control or perform additional procedures to determine whether the lack of segregation of duties had allowed fraud to occur. Occurrences of duplicate invoices also would increase the risk of fraud. However, DCAA's audit program for testing contractor billing system controls does not include specific procedures to test for duplicate contractor invoices. We found evidence of testing related to duplicate invoices in only 2 of the 37 internal control audits we reviewed. Moreover, in the audit described below, DCAA FAO managers ordered an auditor to ignore significant fraud risks during an audit. [Text box: DCAA FAO and Region Management Prevented an Auditor from Pursuing Significant Fraud Risks during a Billing System Audit; During a fiscal year 2003 incurred cost audit of a major defense contractor, a DCAA Central Region auditor learned of a fraud investigation initiated by the Army's Criminal Investigative Division (CID) in response to allegations of contractor fraud reported in August 2002. In July 2004, during a billing system audit of the same contractor, the auditor contacted the Army CID investigator to discuss the ongoing fraud investigation and learned that the fraud related to improper billings. As a result of this elevated fraud risk, the auditor requested several nominal increases in budget hours to perform additional testing to determine the extent of the fraud. The auditor had prior DOD contract administration experience and intended to use this experience in applying her audit testing procedures. After approving increases in budgeted hours for this assignment, the regional audit manager told the auditor that her concerns were not valid and to remove her "contracting hat." Eight months later, on April 28, 2005, the auditor submitted a draft audit report to her supervisor. She concluded that the contractor's billing system was inadequate--a finding that would have resulted in the contractor losing its direct-billing privileges. The auditor noted several deficiencies and concerns, including (1) the lack of billing policies and procedures, (2) a lack of training for contractor employees responsible for preparing invoices, (3) indications that the contractor may have billed the government for unapproved and unfunded work, and (4) evidence of an ongoing criminal investigation by the Army CID. After reviewing the report, the supervisor and FAO manager directed the auditor to change the opinion from inadequate to inadequate-in-part because the auditor had not identified any excess or unallowable costs. The audit report, issued on August 31, 2005, reported an inadequate-in-part opinion and combined the first two deficiencies, reporting a total of three significant deficiencies. However, DCAA did not remove the contractor from the direct-bill program, whereby contractors are authorized to submit invoices directly to a government paying office without prior review. The auditor assigned to the original audit was also assigned to the billing system follow-up audit, but she was subsequently removed from the follow-up audit because, according to her supervisor, she was documenting her audit in too much detail. In January 2006, during the follow-up audit, Army CID concluded its fraud investigation. The contracting officer's technical representative (COTR) and several contractor employees were convicted of fraudulently billing the government using the billing system that DCAA later deemed adequate. The investigation found that the COTR and the contractor employees were charging the government for travel to contract-related conferences and arranging the trips so they could attend a NASCAR race at government expense. They took government cars on the trips and various contractor employees each charged the government for use of their personal cars, with the COTR approving the travel vouchers. In addition, the COTR had contractor employees cut scrap lumber on government land and stack it at his home for use as firewood. The government was billed for the contractor employees' time on behalf of the COTR. In the January 2006 settlement, which totaled over $2.8 million, the COTR and contractor employees paid fines and restitution, and the COTR also served jail time. The Army CID Special Agent in charge of the fraud investigation told us that he had tried on numerous occasions to get the DCAA FAO manager to stop issuing incurred cost audit reports with "clean" opinions because the opinions would be contradicted by the findings in the ongoing fraud investigation. The FAO issued the 2002 incurred cost audit report on January 5, 2005, stating its opinion that except for the qualification that the ongoing fraud investigation had developed information which may impact the costs and transactions in this report, the claimed direct costs are acceptable and are provisionally approved, pending final acceptance. DCAA did not include a cautionary note or similar qualification in the billing system audit report. In September 2006, DCAA reported an adequate ("clean") opinion in the follow-up audit report on the contractor's billing system controls without performing work to confirm that the contractor's billing system policies and procedures were effectively implemented. Following GAO's review of these audits, on November 20, 2008, DCAA rescinded both reports because the audit documentation did not support the reported opinions and initiated a new audit of the contractor's billing system controls. [End of text box] Insufficient Documentation of Sampling and Testing Methodology: Testing is a critical auditing procedure that allows auditors to determine whether controls are operating effectively. Although some testing can involve statistical samples, such samples are not required under GAGAS. Instead, GAGAS require that auditors prepare attest documentation in sufficient detail to enable an experienced auditor, having no previous connection to the audit, to ascertain from the attest documentation that the evidence supports the auditors' significant judgments and conclusions. Under GAGAS, attest documentation should contain the objectives, scope, and methodology of the attestation engagement, including any sampling and other selection criteria used.[Footnote 126] Of the 37 internal control audits we reviewed, 27 audits did not contain workpaper documentation to demonstrate that the auditors' nonstatistical samples met these requirements. for example: * On one billing system audit, the auditor performed testing on two vouchers. The auditor did not document how he selected the two vouchers, and he did not document the population of contractor vouchers in the workpapers or the basis for his judgment on selecting the two vouchers for testing. When we asked the auditor why he selected two most recent vouchers for testing and did not document the voucher population, the auditor told us it was because "the file size was too large," and he saves the population files on his desktop computer. * On a billing system audit of one of the five largest DOD contractors we asked the auditor why he tested only one voucher to assess the contractor's controls for subcontractor accounting and billing. The auditor said this was reasonable because DCAA "had tested so many vouchers before." Other workpaper documentation noted testing was not performed on the direct-bill section of the audit program. When we asked the auditor why these procedures were not performed, the auditor told us that testing was performed by another FAO when the contractor implemented a new system 2 months earlier, and he decided not to do testing again because "the contractor would not appreciate it"--an indication of an auditor independence problem. Moreover, tests of new billing systems focus on data processing controls and would not take the place of tests of invoices for compliance with CAS, FAR, and contract terms. Although the CAM includes guidance on sufficient testing,[Footnote 127] auditors appeared to follow general guidance throughout the manual that advises auditors to use their judgment "to 'test check' a procedure, to make verifications 'on a selective basis,' or to review a 'representative number' of transactions or items." Several auditors, field office managers, and DCAA headquarters officials told us that they believed "spot checks" were sufficient testing to conclude on controls overall and they did not believe they were required to document their sampling plans. [Text box: DCAA Relied on Faulty Auditor Judgment to Approve Contractor Controls; An Eastern Region auditor performed minimal testing in an audit of controls over indirect and other direct cost for a business segment of one of the top five DOD contractors that billed the government for about $1 billion during 2006. The auditor did not use statistical sampling or test a representative selection of accounts payable transactions. Instead, without documenting the reasons for his judgments, the auditor tested 6 of 16,000 accounts payable transactions ($86 of $50 million), 3 of 4,500 travel transactions ($2,700 of $15 million), and 3 of 1,600 interdivisional transactions ($5,000 of $16 million). On September 27, 2006, DCAA reported an adequate opinion on the contractor's controls over direct and indirect costs. However our review of the audit workpapers revealed: * no explanation of why so few transactions were tested; * no rationale for why transactions selected for testing covered the months of May through July 2005, when transactions occurred throughout the year, or; * how the auditor concluded that the system was adequate based on testing 12 out of about 22,000 transactions. When we asked the auditors to explain the basis for their selection of transactions used for testing, FAO management said the selection was based on auditor judgment; implying auditors could use their professional judgment without the need to meet any specific criteria in doing so. GAGAS section 3.34 (GAO-03-673G) states that auditors should consider the need to protect the public interest when making professional judgments. GAGAS section 6.02a requires auditors to perform sufficient testing to support audit conclusions and opinions on controls. Determining what is sufficient testing requires auditors to determine an appropriate sample size considering risks, expectation of misstatements or deviations, and materiality, and select a representative sample from the population, meaning that all transactions have a known chance of being selected. GAGAS section 6.24 a. and c. require auditors to document the sampling plan and auditor judgments made in sampling and testing. This audit did not meet these GAGAS requirements. [End of text box] Insufficient Evidence to Support Audit Conclusions and Opinions: We found that audit procedures for most of the 37 internal control audits we reviewed documented the design of controls but did not test the implementation of controls. As a result, the audits lacked sufficient evidence to support audit opinions that covered both the design and implementation of controls. GAGAS for examination-level attestation engagements require that sufficient evidence be obtained to provide a reasonable basis for the conclusion that is expressed in the report.[Footnote 128] GAGAS state that attest documentation serves to (1) provide the principal support for the auditor's report, (2) aid auditors in conducting and supervising the attestation engagement, and (3) allow for the review of the quality of the attestation engagement. The preparation of attest documentation should be appropriately detailed to provide a clear understanding of its purpose and source and the conclusions the auditors reached, and it should be appropriately organized to provide a clear link to the findings, conclusions, and recommendations contained in the auditors' report.[Footnote 129] Overall, we found that 33 of the 37 internal control audits did not include sufficient testing of internal controls to support auditor conclusions and opinions. Our review of audit workpapers often found that only two, three, or sometimes five transactions were tested to support audit conclusions, for example: * On several audits, DCAA concluded that a contractor had adequate controls for removing system access for terminated or transferred employees. However, the auditors did not document the employee population from which individual employees were selected for testing system access, or the methodology used to select them. On none of these audits did we see evidence that DCAA auditors checked alphabetical listings of individuals having system access to lists of current personnel to confirm that access was removed when employees transferred or left the company. Without documentation of sampling and testing methodologies, there is no way to ascertain how the auditors came to their conclusions that controls were adequate or that sufficient testing was done to support audit conclusions. * For many controls, DCAA did not perform any testing at all. For example, at least 6 of the 9 accounting audits we reviewed did not include procedures for testing contractor segregation of allowable and unallowable cost; 20 of 22 billing system audits we reviewed did not include tests to identify duplicate invoices, and 10 of the 22 billing system audits of contractors that relied on manual procedures to prepare invoices from accounting system data queries did not check for compensating controls. For one audit, DCAA issued an adequate opinion on the accounting system for a major DOD contractor after performing a walkthrough of the accounting process and interviewing two employees. [Text box: Adequate Opinion on Contractor Billing System Was Based on Spot Checks of 4 Vouchers Generated on the Same Day; A Mid-Atlantic Region auditor used interviews with contractor staff and limited testing as evidence that billing system controls were adequate for a DOD contractor with about $40 million in annual government sales. Workpapers documenting audit procedures on key internal controls referred to "discussions with the contractor" rather than independent auditor verification, including (1) verification of periodic reviews of contractor policies and procedures, (2) implementation and effectiveness of policies and procedures, (3) frequency and sufficiency of the contractor's management reviews, (4) timely processing of offsets, and (5) exclusion of non-billable items from government billings. Although the audit was performed from November 2004 through July 2005, according to the workpapers, the auditor tested a nonstatistical selection of four vouchers (invoices) totaling $2.3 million that were all processed on the same day--February 28, 2005. The workpapers contained no documentation on the population of invoices or the basis for selecting four vouchers for testing that were all processed on the same day out of the 8-month period covered by the audit. GAO also determined that the auditors performed no testing of the contractor's billing system information technology (electronic data processing) controls. As a result, this audit can not be relied on for assurance that the contractor's billing system and related internal control policies and procedures were adequate as of June 16, 2005. [End of text box] Audit Supervision Problems: GAGAS require that assistants (audit staff) be properly supervised and that audit documentation contain evidence of supervisory reviews of the work performed that supports findings, conclusions, and recommendations contained in the report before the report on the attestation engagement is issued.[Footnote 130] Although workpaper documentation for the majority of the 37 audits of contractor internal control systems we reviewed evidenced supervisory review, we found: * A lack of proper documentation of supervisory review in 13 audits. For example, for an Eastern Region accounting system audit, the supervisory auditor who signed the audit report did not review key workpapers related to accounting system transaction processing and transaction testing and cost allocations until 1 to 2 days after the audit report was issued. This was similar to a situation we found in our prior investigation, when supervisors at one DCAA field office frequently reviewed the workpapers for forward pricing reports after the reports were issued. The auditors also performed insufficient testing on this audit. * Audit steps were deleted from the standard audit program in an accounting system audit and a billing system audit after the supervisors approved the audit programs. The supervisors did not ensure that the deleted steps were addressed or that documentation was added to the workpapers to explain the reasons why the related audit procedures were not performed. * For six other audits, audit documentation shows that the supervisors and FAO managers extended the audit time frames while contractors took actions to correct significant deficiencies. The audit reports were issued 1 to 2 years later with adequate ("clean") opinions on controls. Although this raises serious auditor independence and reporting issues because identified deficiencies were not reported, we are highlighting these cases under our discussion of poor supervision to also demonstrate the importance of "tone at the top." [Text box: DCAA Extended Audit and "Scrubbed" Audit Documentation after Contractor Objected to Findings; A DCAA Western Region FAO failed to provide proper supervision of auditors throughout an accounting system audit of one of DOD's five largest contractors working in Iraq. For contractor fiscal year ended December 31, 2004, the contractor reported over $900 million in sales of which 98 percent related to government contracts, including $250 million for work in Iraq. The 2004 audit, which was initiated in November 2003, was transferred among several auditors and at least three supervisors before its completion and August 2006 publication. In September 2005, the contractor objected to draft findings and recommendations that included eight significant deficiencies in the design and operation of the contractor's accounting system, including inadequate system access controls, lack of policies and procedures for segregation of duties, lack of periodic reconciliations of cost accounts to the general ledger, and insufficient cost ledger information on total base costs by contract and cost elements for applying indirect rates. The contractor stated that the auditors did not fully understand the new policies and procedures that were just being developed for the fast track effort in Iraq. Following the contractor's objections, the auditors revised and deleted some workpapers and created new workpapers. GAO's review of the audit documentation identified several workpapers that were indexed to supporting documentation that no longer existed. Further, the auditors told GAO that because they had difficulty finding support for Iraq vouchers, they relied on voucher reviews performed under other DCAA audits. GAO also found evidence in the audit documentation that the final supervisor instructed the final lead auditor to insert the signature of a prior supervisor on an electronic workpaper after it had been revised, thereby making it appear that the prior supervisor had approved the workpaper revisions. On August 31, 2006, after "scrubbing" the audit documentation at the supervisor's request, dropping five significant deficiencies and downgrading three significant deficiencies to suggestions for improvement, DCAA reported an adequate ("clean") opinion on the contractor's accounting system. Waiting to review audits with significant deficiencies until the end of the job after the work has been completed, raises questions about proper and timely supervision. The audit supervisor, who authorized the electronic recording of the prior supervisor's name on the audit documentation and supervised the issuance of the audit report, was subsequently promoted to be the Western Region Quality Assurance Manager, where he went on to act as a quality control check over thousands of audits--including several of the audits investigated in GAO's prior work. Following GAO's review, DCAA rescinded the audit report on December 2, 2008. [End of text box] Reporting Problems: Audit reports are DCAA's principal work product. According to GAGAS, audit reports should, among other criteria, (1) identify the subject matter being reported, the criteria used to evaluate the subject matter, the conclusion or opinion, and state that the opinion was as of a certain date; (2) include a statement of the nature and scope of the work performed and state that the audit was performed in accordance with GAGAS; (3) disclose any reservations about the engagement, including any scope limitations; (4) state the intended use of the report, if limited; and (5) state the time frame[Footnote 131] covered by the audit. Our review of audit documentation and DCAA final audit reports determined that none of the 37 DCAA reports on contractor systems internal controls met these reporting standards, for example: * The reports did not cite the specific criteria used in individual audits. Criteria represent the laws, regulations, contracts, grant agreements, standards, measures, expected performance, defined business practices, and benchmarks against which performance is compared and evaluated. Criteria identify the required or desired state or expectation with respect to the program or operation and provide a context for evaluating evidence and understanding the findings. [Footnote 132] Instead, the DCAA reports uniformly used boilerplate language to state that DCAA audited for compliance with the "FAR, CAS, DFARS, and contract terms." As a result the user of the report does not know the specific Federal Acquisition Regulation (FAR), Cost Accounting Standards (CAS), or contract terms used as criteria to test contractor controls. This makes it very difficult for users of the reports to determine whether the report provides the level of assurance needed to make contract management decisions. In addition, audit documentation for many of the audits we reviewed did not identify the audit work performed to provide assurance that contractors complied with specific requirements in CAS, FAR, DFARS, or contract terms. * Six of the 37 audit reports were not issued at the time[Footnote 133] the work was completed. These reports were issued from 8 months to over 2 years after the audits were completed. Frequently, we found that the delays were the result of serious findings, which led DCAA to withhold issuance of the report while the contractor addressed the problems. Because testing was not updated or was not sufficiently updated, the reported audit opinions, which related to controls at the time the reports were issued, were not adequately supported and may have been inaccurate. * The audit reports stated the period during which the audit was performed but did not disclose the scope and timing for tests of vouchers, transactions, or control attributes. Some tests covered a few days in only one month or a 3-month period and did not test controls across the year audited. As a result, testing did not support the reported audit opinions as of the report dates. * Contractors imposed restrictions on the scope of four audits by denying DCAA access to certain records. The access-to-records issues were not fully resolved or disclosed by the auditors. * The scope of 33 audits was limited by DCAA imposed, or implied, restrictions, including inadequate audit resources, unclear audit guidance on nature and extent of testing, and time constraints that prevented auditors from performing sufficient work to support reported opinions on contractor internal controls. DCAA officials told us that DCAA does not have sufficient resources to perform full-scope audits of contractor internal controls. Failure to issue reports when sufficient evidence has been obtained to support an auditor's conclusion puts decision makers at risk of relying on out-dated or inaccurate information. Also, when DCAA auditors do not perform the scope of work necessary to support the reported audit opinions, the audit reports provide a false level of assurance. Following our discussion of these audits with DCAA headquarters officials, DCAA rescinded 4 of the 6 audit reports that did not accurately relate the period of testing to the audit opinion, and it rescinded 18 audit reports where the scope of work did not support the audit opinions. The discussion below describes a particularly egregious example of this problem. [Text box: Two Years after Testing Controls, DCAA Reported the Results of an Audit of a Multibillion Dollar Contractor's Billing System; In July 2003, DCAA initiated a billing system audit of a contractor doing business in Iraq with sales of $6.3 billion at the two divisions under audit. More than 2 years after performing test procedures and after spending 1,025 hours on the audit, the FAO issued an opinion that the contractor's billing system controls were adequate as of August 31, 2005, without updating the testing. In 2003, DCAA auditors tested 38 vouchers submitted for payment within a 12-day period. DCAA auditors identified numerous billing errors, including two instances where billings did not comply with the Federal Acquisition Regulation (FAR). On August 12, 2004, DCAA auditors prepared a draft report with an adequate opinion and three suggestions for improvement. DCAA auditors did not perform testing in 2004 or in 2005 despite the number of errors found as a result of limited test procedures performed in 2003. As a result, the evidence does not support the opinion that the contractor's billing system controls are adequate as of August 31, 2005. Additionally, there is no evidence in the workpapers that the contractor resolved the errors DCAA identified and the underlying system deficiencies that caused those errors. This is of special concern because billing errors and system deficiencies at this contractor put multiple agencies at risk. For example, this contractor does work not only for DOD but also for the Departments of Agriculture, Commerce, Interior, and NASA, and several other agencies. The lead auditor told us that this audit was delayed because numerous auditors were assigned over the course of this audit, and the contractor's work in Iraq took precedence over this audit. However, we found no evidence supporting a decision not to issue this report when the testing was completed in 2003. Following GAO's review, DCAA rescinded the audit report on April 7, 2009. [End of text box] [End of section] Appendix II: DCAA Does Not Perform Sufficient Work to Identify and Collect Contractor Overpayments: DCAA performs assignments that are designed to test various contractor costs as allowable, reasonable under the related contracts, the Federal Acquisition Regulation (FAR), and Cost Accounting Standards (CAS). Although DCAA uses the term "audit" generically, some of these assignments are audits and other assignments relate to financial and advisory services. We reviewed 32 cost-related assignments[Footnote 134] performed by seven geographically disperse field audit offices (FAO) across the five DCAA regions (the same offices as in appendix I) to assess whether (1) the tests of contractor costs, billings, and payments were effective in identifying overpayments, billing errors, and unallowable cost[Footnote 135] and (2) DCAA identified and reported unallowable and unsupported costs, overpayments, and billing errors so that the government was in a position to collect or recover improper costs and billings through refunds, contract adjustments, or offsets. The 32 DCAA cost-related assignments we reviewed included 16 paid voucher reviews, 10 overpayment assignments, 4 incurred cost audits, and 2 request for equitable adjustment (REA) audits.[Footnote 136] Although DCAA performs incurred cost and REA audits as engagements in accordance with generally accepted auditing standards (GAGAS), DCAA does not consider paid voucher reviews or overpayment audits to be GAGAS assignments. DCAA performed the paid voucher reviews to assess the accuracy of contractor billings to support decisions to approve contractors for participation in the direct-bill program,[Footnote 137] whereby the contractor submits invoices directly to a federal agency paying office without government review of the invoices prior to payment. Overpayment assignments review contractor controls for identifying and refunding, offsetting, or adjusting contract overpayments and billing errors. Similarly, DCAA performs overpayment assignments at the request of contracting officers to determine (1) whether contractor controls are effective in identifying overpayments made by disbursing officers or over billings and billing errors made by contractors and (2) if contractors are making timely refunds, offsets, or adjustments. At the time the 32 cost-related assignments were performed, FAR 52.232-25(d) imposed a requirement on contractors to immediately notify the contracting officer and request instructions for disposition of any overpayment when the contractor becomes aware of a duplicate or overpaid contract financing or invoice payment.[Footnote 138] Also, FAR 32.604(b)(4) provides that contractors shall repay debts under a demand letter within 30 days, except for certain debts covered by specific terms of the contract.[Footnote 139] This time period is incorporated into most contracts under FAR Clause 52.232-17(a). We found that DCAA auditors are not consistent when assessing the timeliness of refunds and offsets. Specifically, although DCAA's overpayment work program cites 30 to 60 days after the overpayment occurred as timely,[Footnote 140] some DCAA auditors considered 90 days as timely which effectively minimized the impact on the contractors' cash flow. We also found limited testing in the four incurred cost audits we reviewed. DCAA considers incurred cost audits to be GAGAS attestation engagements. Incurred cost audits examine contractors' annual claims for payment of cost incurred. DOD contracting officers rely on DCAA incurred cost audits to approve contractor claims for payment.[Footnote 141] DCAA incurred cost audits and proposal audits are the source of most DCAA questioned costs and dollar recoveries. Dollar recoveries are based on contracting officer agreement with DCAA questioned costs. [Footnote 142] DOD contracting officers are responsible for enforcing DCAA recommendations to disallow questioned cost. Figure 3 provides a comparison of costs questioned by DCAA auditors and questioned costs sustained (recovered) by DOD contracting officers. Figure 3: DCAA Questioned Costs and Amounts Sustained by Contracting Officers: [Refer to PDF for image: multiple line graph] Fiscal year: 2000; Questioned costs: $5.0 billion; Questioned costs sustained: $3.4 billion. Fiscal year: 2001; Questioned costs: $6.8 billion; Questioned costs sustained: $4.6 billion. Fiscal year: 2002; Questioned costs: $4.0 billion; Questioned costs sustained: $3.0 billion. Fiscal year: 2003; Questioned costs: $4.2 billion; Questioned costs sustained: $2.8 billion. Fiscal year: 2004; Questioned costs: $4.7 billion; Questioned costs sustained: $3.5 billion. Fiscal year: 2005; Questioned costs: $5.5 billion; Questioned costs sustained: $3.2 billion. Fiscal year: 2006; Questioned costs: $5.3 billion; Questioned costs sustained: $3.4 billion. Fiscal year: 2007; Questioned costs: $4.9 billion; Questioned costs sustained: $3.1 billion. Fiscal year: 2008; Questioned costs: $6.7 billion; Questioned costs sustained: $4.2 billion. Source: GAO analysis of DCAA data. [End of figure] For one of the four incurred cost audits we reviewed, DCAA rescinded the related accounting system audit report in response to concerns we identified with that report. For a second incurred cost audit we reviewed, DCAA rescinded the related billing system report. Risk assessments for determining the nature, extent, and timing of testing for incurred cost audits are based in part on the results of accounting and billing system audits. Therefore, a rescission of an accounting or billing system audit would call into question the risk assessment performed for the related incurred cost audits. The case study examples in table 6 illustrate significant problems we identified with the DCAA cost-related assignments we reviewed. As previously discussed, the level of testing in these assignments was not sufficient to identify all potential contractor billing errors and overpayments. Table 6: Case Studies of Problem DCAA Cost-Related Assignments: Region: Central; Type of assignment: Paid voucher review (2006); Non-GAGAS; Details of review: * For this review of paid contractor invoices, the auditor relied on the results of DCAA's 2005 billing system audit and did not test any invoices. The workpapers stated that the auditor also relied on the results of the 2005 paid voucher assignment. However, that assignment did not test any 2006 invoices; * Further, as a result of GAO's work, DCAA had rescinded the 2005 billing system audit report on February 10, 2009; * As a result, there is no audit support for DCAA's approval for this contractor to directly bill the government. Region: Central; Type of assignment: Paid voucher; review (2005); Non-GAGAS; Details of review: * In planning this work, the auditor improperly assessed risk as low and deleted several steps from the standard "audit" program; * The auditor did not identify the population of vouchers (invoices) and selected two invoices for testing, but only tested one of them; * The auditor tested one invoice to see if the payment received by the contractor matched the amount billed; * On January 23, 2006, DCAA issued a Memorandum for the Record, stating "reliance can be placed on the contractor's procedures for preparation of interim vouchers. Accordingly, the contractor has met the criteria for continued participation in the direct billing program." Region: Western; Type of assignment: Paid voucher; review (2005); Non-GAGAS; Details of review: * Without documenting the population of vouchers or the total dollars billed during the contractor's fiscal year, the auditor tested 8 of 734 vouchers issued from April 16, 2004, through March 25, 2005; * The supervisor incorrectly directed the auditor to test a final voucher. Paid voucher assignments focus on interim vouchers as a basis for making direct-bill decisions. Final vouchers are submitted to close out a contract; * The auditor did not identify any errors in the vouchers tested; * On September 30, 2005, the auditor prepared a Memorandum for the Record, stating that "continued reliance can be placed on the contractor's procedures for preparation of interim vouchers. Accordingly, the contractor has met the criteria for continued participation in the direct billing program." Region: Eastern; Type of assignment: Paid voucher review (2004); Non-GAGAS; Details of review: * Although the contractor generated $1.1 billion in annual billings to the government, the auditor assessed risk as low for this assignment. Without documenting the basis for the risk assessment, the auditor judgmentally selected 3 vouchers totaling $88,000 for testing out of a total of 222 vouchers submitted to the government for payment from March 2003 through February 2004; * The auditor tested the first voucher selected and performed limited testing on the remaining 2 vouchers; * The workpapers do not include any evidence to show that the auditor performed most of the audit steps required in the standard audit program; * Despite limited testing, on March 31, 2004, DCAA prepared a memorandum for the record, stating "continued reliance can be placed on the contractor's procedures for the preparation of interim vouchers" and "the contractor has met the criteria for continued participation in the direct billing program." Source: GAO analysis of DCAA audit documentation. [End of table] [End of section] Appendix III: Objectives, Scope, and Methodology: Pursuant to a request from the Chairman and Ranking Member of the Senate Committee on Homeland Security and Governmental Affairs, we conducted an agencywide performance audit to assess the effectiveness of Defense Contract Audit Agency (DCAA) audits for helping to assure that prices paid by the government for needed goods and services are fair and reasonable and that contractors are charging the government in accordance with applicable laws, regulations, cost accounting standards, and contract terms. The overall objectives of our work were to (1) conduct a broad assessment of DCAA's management environment and quality assurance structure, (2) evaluate DCAA corrective actions in response to our prior investigation[Footnote 143] and DOD Comptroller/ CFO "tiger team" and Defense Business Board (DBB) studies, and (3) identify potential legislation and other actions that could improve DCAA's effectiveness and independence. To address our first objective, we evaluated DCAA's contract audit guidance and policies and its quality assurance program and assessed the quality of a nationwide selection of DCAA audits. We evaluated the results of internal DCAA audit quality assurance reviews on audits issued from fiscal year 2003 through 2008. We also reviewed a total of 69 DCAA audits and cost-related assignments.[Footnote 144] In reviewing DCAA audits, we used generally accepted government auditing standards as our criteria.[Footnote 145] The 69 DCAA audits and cost-related assignments we reviewed included 37 audits of contractor internal controls and 32 cost-related audits and assignments. We did not assess a statistical sample of DCAA audits. Rather, we focused on DCAA offices that reported predominately adequate, or "clean," opinions on audits of contractor internal controls over cost accounting, billing, and cost estimating systems issued in fiscal years 2005 and 2006.[Footnote 146] We selected DCAA offices that reported predominately adequate ("clean") opinions on contractor systems and related internal controls because contracting officers rely on these opinions for three or more years to make decisions on pricing and contract awards, and payment. For example, audits of estimating system controls support negotiation of fair and reasonable prices.[Footnote 147] Also, the Federal Acquisition Regulation (FAR) requires contractors to have an adequate accounting system prior to award of a cost-reimbursable or other flexibly-priced contract.[Footnote 148] Billing system internal control audit results support decisions to authorize contractors to submit invoices directly to DOD and other federal agency disbursing offices for payment without government review.[Footnote 149] In addition, DCAA uses the results of internal control audits to assess risk and plan the nature, extent, and timing of tests for other contractor audits. When a contractor has received an adequate opinion on its systems and related controls, DCAA would assess the risk for subsequent internal control and cost-related audits as low and would perform less testing on these audits. Using this approach, we identified seven geographically disperse DCAA field offices within the 5 DCAA regions and targeted 39 audits of contractor cost accounting, billing, and estimating system controls issued during fiscal years 2004 through 2006 for review.[Footnote 150] These were the most recent completed fiscal years at the time we initiated our audit. Two of the 39 internal control audits we identified were performed as assist audits to a billing system audit and we considered them as part of the billing system audit we reviewed. Therefore, we reviewed a total of 37 audits of contractor internal controls for compliance with GAGAS and DCAA policy. We also considered whether DCAA adequately applied internal control standards in its audits that are applicable to the private sector.[Footnote 151] We did not review classified audits performed by DCAA's field detachment office. Although our selection of the seven offices and 37 internal control audits was not statistical, it represented about 9 percent of the total 76 DCAA offices that issued audit reports on contractor internal controls and nearly 18 percent of the 40 offices that issued 8 or more reports on contractor internal controls during fiscal year 2006. Of the 37 internal control audits we reviewed, 32 reports were issued with adequate opinions and 5 reports were issued with inadequate- in-part opinions. Table 7 summarizes the number and types of contractor internal control audits we reviewed for seven FAOs across the 5 DCAA regions. Table 7: Summary of DCAA Audits Reviewed for GAGAS Compliance: Region/FAO: Northeast: FAO #1; Type of internal control audit: Accounting system: [Empty]; Indirect & other direct cost: [Empty]; Billing system: 3; Estimating system: 1; Total internal control audits: 4. Region/FAO: Mid-Atlantic: FAO #2; Type of internal control audit: Accounting system: 2; Indirect & other direct cost: [Empty]; Billing system: 3; Estimating system: [Empty]; Total internal control audits: 5. Region/FAO: Eastern: FAO #3; Type of internal control audit: Accounting system: 1; Indirect & other direct cost: 1; Billing system: 3; Estimating system: 1; Total internal control audits: 6. Region/FAO: Central: FAO #4; Type of internal control audit: Accounting system: 2; Indirect & other direct cost: [Empty]; Billing system: 2; Estimating system: 1; Total internal control audits: 5. Region/FAO: Central: FAO #5; Type of internal control audit: Accounting system: 2; Indirect & other direct cost: [Empty]; Billing system: 5; Estimating system: [Empty]; Total internal control audits: 7. Region/FAO: Western: FAO #6; Type of internal control audit: Accounting system: [Empty]; Indirect & other direct cost: 1; Billing system: 3; Estimating system: 1; Total internal control audits: 5. Region/FAO: Western: FAO #7; Type of internal control audit: Accounting system: 2; Indirect & other direct cost: [Empty]; Billing system: 1; Estimating system: 2; Total internal control audits: 5. Total: Type of internal control audit: Accounting system: 9; Indirect & other direct cost: 2; Billing system: 20; Estimating system: 6; Total internal control audits: 37. Source: GAO analysis of DCAA management information system data. [End of table] At the same seven DCAA field offices, we selected 34 cost-related assignments performed during the same period as the internal control audits we reviewed and analyzed supporting documentation to determine whether the assignments included sufficient testing to assess whether (1) the tests of contractor costs, billings, payments were effective in identifying overpayments, billing errors, and unallowable cost[Footnote 152] and (2) DCAA reported overpayments, billing errors, and unallowable and unsupported costs, so that the government was in a position to recover improper payments through refunds, contract adjustments, or offsets and avoid payment of unsupported and unallowable costs. Upon reviewing documentation for the 34 cost-related audits, we determined that one of these assignments covered the risk assessment portion of an incurred cost audit and was not a complete audit. Documentation for a second assignment to test for overpayments was terminated and the audit procedures were rolled into a billing system audit. Consequently, as shown in table 8, we reviewed a total of 32 cost-related DCAA assignments. These assignments included paid voucher reviews and overpayment control assignments and audits of requests for equitable adjustment (REA)[Footnote 153] and contractor incurred cost claims. Table 8: Summary of DCAA Cost-Related Assignments Reviewed: Region/FAO: Northeast: FAO #1; Type of Assignment: Paid Voucher review: [Empty]; Over payment assignment: 1; REA audit: 2; Incurred Cost audit: 2; Total assignments: 5. Region/FAO: Mid-Atlantic: FAO #2; Type of Assignment: Paid Voucher review: 4; Over payment assignment: 1; REA audit: [Empty]; Incurred Cost audit: 1; Total assignments: 6. Region/FAO: Eastern: FAO #3; Type of Assignment: Paid Voucher review: 3; Over payment assignment: 3; REA audit: [Empty]; Incurred Cost audit: [Empty]; Total assignments: 6. Region/FAO: Central: FAO #4; Type of Assignment: Paid Voucher review: 1; Over payment assignment: 1; REA audit: [Empty]; Incurred Cost audit: [Empty]; Total assignments: 2. Region/FAO: Central: FAO #5; Type of Assignment: Paid Voucher review: 5; Over payment assignment: 2; REA audit: [Empty]; Incurred Cost audit: [Empty]; Total assignments: 7. Region/FAO: Western: FAO #6; Type of Assignment: Paid Voucher review: 3; Over payment assignment: 1; REA audit: [Empty]; Incurred Cost audit: [Empty]; Total assignments: 4. Region/FAO: Western: FAO #7; Type of Assignment: Paid Voucher review: [Empty]; Over payment assignment: 1; REA audit: [Empty]; Incurred Cost audit: 1; Total assignments: 2. Total: Type of Assignment: Paid Voucher review: 16; Over payment assignment: 10; REA audit: 2; Incurred Cost audit: 4; Total assignments: 32. Source: GAO analysis of DCAA management information system data. [End of table] The details of our assessments of DCAA audits of contractor internal control systems and cost-related audits and assignments are included in appendixes I and II, respectively. Examples of our findings are included in the body of this report to help illustrate the effect of our findings related to DCAA's management environment. To assess DCAA's management environment and quality control system, we reviewed DCAA's mission statement, strategic plan, performance metrics, quality assurance program, audit planning and policy guidance, and human capital management. We evaluated the results of internal DCAA audit quality assurance reviews on audits issued from fiscal year 2004 through 2008. We used requirements in the Government Performance and Results Act,[Footnote 154] GAGAS,[Footnote 155] and GAO's Standards for Internal Control in the Federal Government[Footnote 156] as our criteria. We analyzed the findings, conclusions, and recommendations in the DOD Inspector General's (IG) 2007 report on its oversight review of DCAA, [Footnote 157] which serves the purpose of a peer review. We did not review DOD IG documentation for the oversight review. In assessing the DOD IG peer review conclusions and opinion, we considered the inconsistencies between the findings and recommendations in the IG report. In addition, we considered the results of our analysis of DCAA audits in our prior investigation; our review of the 69 DCAA audits and related assignments covered in this report; the results of DCAA's internal quality assurance reviews; and DCAA's actions to rescind 80 audit reports. To achieve our second objective, we reviewed the status of several key actions that DCAA initiated as a result of our earlier investigation, including efforts to: * revise DCAA's mission statement and strategic plan to focus on protecting the public interest; * change performance metrics to focus on audit quality instead of performing large quantities audits; * end DCAA involvement with integrated product teams, which we identified as an impairment to DCAA's independence; * improve audit quality by revising audit policy guidance and realigning DCAA's audit quality assurance structure; and: * update training courses to reflect changes in DCAA's mission, metrics, and audit policy. Although the October 2008 Defense Business Board report recommended that the Secretary of Defense revise DCAA's mission statement to focus on protecting the public interest, at the time we completed our work in July 2009, DCAA's mission statement had not yet been revised. To assess changes in performance metrics, we analyzed DCAA's new metrics and determined whether changes made in September 2008 were effective in shifting DCAA focus from report production to performing quality audits and if the new metrics had been integrated into DCAA's performance plans, auditor expectations, and performance appraisal standards. In addition, we made selected calls to one or more auditors in 15 selected DCAA offices that were separate from the offices we visited to review audit documentation and interviewed auditors about their experience with changes in DCAA policies and performance metrics. We also considered 34 additional hotline allegations we received from auditors across the 5 DCAA regions after our investigative report was issued. We used GAGAS criteria[Footnote 158] to assess the effectiveness of DCAA policy changes and DCAA's centralization of the audit quality function aimed at improving auditor independence and audit quality. We used GAO's Internal Control Standards as our criteria for assessing DCAA's management environment, culture, need for a risk-based audit approach, and human capital practices. To achieve our third objective to identify potential legislative and other actions to improve DCAA's effectiveness, we considered DCAA's current role and responsibilities; the framework of statutory authority for auditor independence in the Inspector General Act of 1978, as amended;[Footnote 159] best practices of leading organizations that have made cultural and organizational transformations; our past work on DCAA organizational alternatives;[Footnote 160] GAGAS criteria for auditor integrity, objectivity, and independence; and GAO's Standards for Internal Control[Footnote 161] on managerial leadership and oversight. We identified potential short-term and longer term legislative actions and organizational changes that could enhance DCAA's effectiveness and independence. Throughout our audit we met with the DCAA Director and DCAA headquarters policy, quality assurance, and operations officials and DCAA Region and FAO managers, supervisors, and auditors. We also met with DOD Office of Inspector General (OIG) auditors responsible for DCAA audit oversight and DOD IG hotline office staff. We assessed the reliability DCAA data used in our work by reviewing DCAA procedures for assuring the reliability of reported performance data, discussing the compilation and use of these data with DCAA operations personnel, and performing analytical procedures to determine the reliability of specific data used in our analysis. For example, we determined that DCAA assignments initiated in one year and completed in the second year were double counted. We eliminated duplicate records from data used for our analysis. We also met with the former DOD Comptroller/CFO to discuss plans for the Office of Comptroller/CFO and Defense Business Board reviews, and we continued to meet with and obtain information from the new DOD Comptroller/CFO and his staff. We also met with the Comptroller's new DCAA Oversight Committee, which includes the Auditors General of the Army, the Navy, and the Air Force; the DOD Director of Defense Procurement and Acquisition Policy; and the DOD Deputy General Counsel for Acquisition. We conducted this performance audit from August 2006 through December 2007, at which time we suspended this work to complete our investigation of hotline complaints regarding audits performed at three DCAA field offices. We resumed our work on this audit in October 2008 and performed additional work through July 2009 to evaluate DCAA's quality assurance program during fiscal years 2007 and 2008, assess DCAA corrective actions, and consider organizational placement options. We conducted our audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for findings and conclusions based on our audit objectives. We performed our investigative procedures in accordance with quality standards set forth by the Council of the Inspectors General on Integrity and Efficiency (formerly the President's Council on Integrity and Efficiency). [End of section] Appendix IV: Comments from the Department of Defense: Note: GAO comments supplementing those in the report text appear at the end of this appendix. Under Secretary Of Defense: Comptroller: 1100 Defense Pentagon: Washington, Dc 20301-1100: September 4, 2009: Mr. Gregory Kutz: Managing Director, Forensic Audits and Special Investigations: Government Accountability Office (GAO): 441 G. St., NW: Washington, DC 20548: Dear Mr. Kutz: This is the Department of Defense (DoD) response to the GAO draft report GAO09-468, "DCAA Audits: Widespread Problems with Audit Quality Require Significant Reform," dated July 31, 2009 (GAO code 195099). We thank you for the opportunity to respond to the GAO draft report and recommendations. Those recommendations directed to the DoD Inspector General will be provided directly by that office under separate cover. The Department concurs with all but one of the GAO recommendations (Enclosure 1). We disagree with some of GAO comments for Congressional consideration (Enclosure 2). Enclosure 3 contains a listing of corrective actions DCAA has implemented since 2008. As part of our review of the GAO draft report, we noted several areas where we believe further clarification and explanation are needed so the circumstances can be completely understood. The clarifying comments are provided in Enclosure 4. Comments from the Director, Defense Procurement and Acquisition Policy are provided as Enclosure 5. The Department and the Defense Contract Audit Agency (DCAA) are committed to taking the necessary corrective actions to address the GAO findings. This office will continue to monitor DCAA to ensure timely completion of critical actions to address the GAO's recommendations. To assist with this monitoring, in March 2009, I established a DCAA Oversight Committee to provide my office advice and recommendations related to the oversight of DCAA. This committee includes Auditors General of the Army, the Navy, and the Air Force; the Director of Defense Procurement and Acquisition Policy; and the DoD Deputy General Counsel for Acquisition and Technology. The committee will assess DCAA's activities, its corrective actions taken to this and other oversight reports, and identify any gaps. I have also assigned a member of my senior staff to assist in the oversight efforts. We acknowledge that the draft GAO report raises serious issues that we need to continue to address. We will continue to improve audit quality, especially for internal control audits. We will reassess the number of audits that DCAA performs and ensure that adequate staffing is available to conduct those audits in accordance with generally accepted government auditing standards (GAGAS). We will ensure that DCAA has adequate independence. We will also ensure that an appeals mechanism is created to provide a process for resolving disputes between DCAA and the contracting organizations. While we acknowledge that continued work on these serious issues is required, we disagree with the suggestion in the GAO report that we have not yet begun to address the weaknesses. The GAO report states that "DoD and DCAA have not yet addressed the fundamental weaknesses in DCAA's mission, strategic plan, metrics, audit approach, and human capital practices that have had a detrimental effect on audit quality." We believe that we have begun to address these issues. The audit assignments covered by this review were completed 3-5 years ago, several years before DCAA implemented a series of corrective actions beginning in late 2008. Although a significant number of short-term actions have been completed, DCAA has many long-term actions still in- process. It may take several years to experience the full effect on the execution of the DCAA audits. Enclosure 3 contains the list of corrective actions DCAA has implemented in late 2008 and to-date in 2009. While we acknowledge GAO's findings are serious, DCAA has and continues to play a vital role in support of the contracting process in ensuring that DoD pays a fair and reasonable price in accordance with applicable regulations in protecting the taxpayers' dollars. In FY 2008 alone, DCAA audits recommended reductions in proposed or billed costs of $17.9 billion (referred to as questioned costs) and $7.2 billion in estimated costs where the contractor did not provide sufficient information to explain the basis of the estimated amounts (referred to as unsupported costs). Additionally, the current Director of Defense Procurement and Acquisition Policy, a key DCAA stakeholder, has stated that he finds DCAA's work critical to the acquisition process (Enclosure 5). DCAA's important work was showcased in three hearings of the Commission on Wartime Contracting in 2009. One hearing focused on contractor business systems, a type of audit that is the subject of the GAO report. Through June 2009, DCAA has cited deficiencies in contractor systems in over half of the 200 audits performed on the contractors performing effort in-theater. The Commission remarked that hundreds of millions of dollars have been recovered as a result of DCAA's audits and that DCAA's efforts were extraordinary considering the performance of audits in a war zone. Several of the Commissioners remarked during a recent hearing that DCAA was the finest audit organization they had worked with and the most forthright and responsive of all the organizations that have interacted with the Commission to date. Several of the key actions DCAA has taken to address DCAA's management environment involved the use of external organizations to guide DCAA through its "cultural transformation." For example, DCAA has engaged the Naval Post Graduate School's Center for Defense Management Reform to assist with the development and execution of various long-term organizational improvement projects. These projects address the following questions: 1. How can DCAA put people first to guide its decisions, actions and values? For example, how can DCAA place an increased emphasis on "soft skills" such as building morale and developing employees (in terms of a broad understanding as well as technical proficiency)? 2. How can DCAA develop leaders to serve the employees and the organization? 3. How can DCAA structure the organization to facilitate compliance with GAGAS, maximize audit results/return on investment (ROI), and better align Agency workload/resources? 4. How can DCAA identify and resolve differing stakeholder expectations with contracting officers, contractors, the public (Congress), and external review organizations? DCAA has also engaged the Army Force Management Support Agency (AFMSA) to assess DCAA's risk-based audit requirements planning process for FY 2010. AFMSA will also assist in translating the requirements planning into staffing needs. One of the major findings reported by the GAO is the lack of sufficient transaction testing to support the conclusions when giving an opinion on contractor internal control systems. We concur that many of the assignment working papers did not reflect the level of testing required by the GAO to opine on the system of internal controls. DCAA has rescinded audit reports and initiated full scope internal control audits at these locations. In light of the GAO's findings, DCAA will be assessing the need to opine on contractor internal controls or whether an opinion on the sufficiency of the contractors' business systems for Government contracting purposes is appropriate. We suggest that the GAO and the DoD Inspector General (IG) work closely with DCAA in its assessment and redesign of testing procedures for opining on contractor business systems. It is worth noting that the additional transaction testing will require an increase in audit staff at DCAA. Some of the increase is already underway through the use of the Defense Acquisition Workforce Development Fund. Using this fund, DCAA has increased staffing by 300 in FY 2009 and plans to increase by an additional 200 each in FYs 2010 and 2011, with a total increase of 700 by the end of FY 2011. However, this may not be enough to accomplish the audits required by regulation in light of the additional testing stipulated by the GAO. We will continue to monitor the staffing situation at DCAA. But to accomplish the significant increase in testing during internal control audits, DCAA will be required to defer lower risk assignments to FYs 2010 and 2011, which could have a negative impact on the timely closing of contracts prior to the cancellation of funds. The overall issue of risk assessment, as outlined in the GAO draft report, is one of the main discussion topics I have assigned to the DCAA Oversight Committee. The Department concurs with the GAO that the root causes for many of the GAGAS noncompliances related to the prior DCAA performance measures. As the GAO acknowledges, in September 2008, DCAA took aggressive action by revising the performance measures to promote quality audits. During FY 2009, DCAA has continually assessed the performance measures by conducting focus groups in several regions (including the DCAA Western Region). In general, the focus groups confirmed that there was much less emphasis on performance measures than in the past. The GAO draft report states that DCAA has taken some actions to improve its quality assurance program; however, it stated that staffing difficulties and other issues have left the outcome of this important initiative uncertain. We disagree. DCAA has been proactive in standing up its new Integrity and Quality Assurance Directorate - essentially a new quality assurance organization that reports directly to the Director/Deputy Director. DCAA has revised its quality assurance program for reporting GAGAS noncompliances and requires audit offices to provide corrective action plans that address all GAGAS noncompliances reported. The Headquarters Quality Assurance organization will follow-up on each systemic noncompliance to ensure the field audit offices have corrected their processes. In July 2009, DCAA was authorized a Senior Executive Service position to lead the Quality Assurance Directorate. We believe the extensive overhaul of the quality assurance function accomplished in FY 2009 will mitigate the prior shortcomings in audit quality cited by the GAO. In January 2009, the DCAA Director established a Senior Advisory Council for Improvement which is led by the Director and comprised of Headquarters senior executives. The Council's primary purpose is to establish and monitor the actions in response to the report issued in January 2009 by the Independent Review Panel to the Defense Business Board and findings from the GAO and DoD IG reviews. The establishment of this council exemplifies the Director's efforts in ensuring key Agencywide actions are managed at the Headquarters level versus the regional level. We believe the establishment of this DCAA Headquarters council, in addition to re-aligning the Quality Assurance Program to report directly to the Director/Deputy Director, results in significant progress toward addressing the GAO's concerns with DCAA's "decentralized structure that has fostered a culture of DCAA region autonomy." The GAO draft report states that the supervisors responsible for deficient audits identified in GAO's prior investigation were promoted even though the GAO investigation disclosed significant GAGAS noncompliances with the audits they supervised. The GAO states that "despite these findings, we found no evidence that supervisors and auditors who did not follow GAGAS and DCAA policy were disciplined, counseled or required to take additional training." DCAA has required several of the supervisors to take additional supervisory and management training courses. They have completed some of these courses and are scheduled to take additional training in the next fiscal year. Based on advice provided by the DoD Washington Headquarters Services Office of General Counsel, DCAA did not take performance or conduct actions against these supervisor;. Additionally, all DCAA employees, including the employees involved in the GAO's 2008 hotline investigation, have completed annual training on auditor "independence" and training in audit quality, including GAGAS, at mandatory standdown days in August 2008 and 2009. We appreciate the recommendations made by the GAO. We believe the implementation of these recommendations along with the many actions that the Department and DCAA have already completed will improve DCAA culture and management environment to ensure taxpayer dollars are protected and DoD audit needs are met. DoD and DCAA leaders are committed to maintaining a strong contract audit function. My point of contact on this matter is Mr. M. Wayne Goff. He can be reached by e-mail at wayne.goff@osd.mil or by telephone at 703-602- 0374. Sincerely, Signed by: Robert F. Hale: Enclosures: As stated: [End of letter] Enclosure 1: GAO Draft Report Dated July 31, 2009: GAO-09-468 (GAO Code 195099): "DCAA Audits: Widespread Problems With Audit Quality Require Significant Reform" Department Of Defense Comments To The GAO Recommendations: Recommendation 1: The GAO recommends that the Secretary of Defense revise the Defense Contract Audit Agency's (DCAA) mission statement to reflect the need for quality contract audits and related nonaudit services that take into account serving the public interest. DOD Response: Concur. The Department is currently seeking final coordination on a revised DCAA mission statement which is included in an updated DoD Directive 5105.36, DCAA. Recommendation 2: The GAO recommends that the Secretary of Defense require the Under Secretary of Defense (Comptroller/CFO) to establish milestones for completing DCAA corrective actions and monitor and regularly report on DCAA progress to assure timely completion of critical actions. DOD Response: Concur. The Department will continue to establish milestones for completing DCAA corrective actions. DCAA already provides USD(C) a monthly status report of the actions taken in response to the recommendations made by the Independent Review Panel of the Defense Business Board as well as recommendations from various GAO and DoD IG reviews. DCAA will expand its improvement plan to incorporate the GAO's recommendations. USD(C) has assigned a senior staff member to assist in oversight and has created the DCAA Oversight Committee, including all three auditors general from the Services, the OSD Deputy General Counsel for Acquisition and Logistics, and the Director of Defense Procurement and Acquisition Policy, to assist in oversight efforts. Recommendation 3: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the Defense Contract Audit Agency (DCAA) in concert with the revised mission statement, develop a Strategic Plan with short-term and long-term outcome-related goals. DOD Response: Concur. DCAA has already started developing a revamped Strategic Plan with short-term and long-term outcome-related goals that is in concert with the proposed mission statement. Assuming the revised mission statement will be approved by about October 2009, DCAA is expected to complete its strategic plan by November 30, 2009 for publication and dissemination to the DCAA workforce. Recommendation 4: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to measure progress in achieving strategic goals, ensure that metrics tie to the revised mission statement and strategic plan and support the agency's annual work plan. DOD Response: Concur. USD(C) will require DCAA to measure progress in achieving strategic goals. Once DCAA's revised mission statement and strategic plan are finalized, USD(C) will require DCAA to ensure that DCAA performance measures tie to the revised mission statement and strategic plan and support the Agency's annual work plan. Recommendation 5: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to consult with DoD stakeholders and engage outside experts to develop a risk-based contract audit approach that identifies resource requirements, and focuses on performing quality audits that meet generally accepted government auditing standards (GAGAS). DOD Response: Partially Concur. The majority of DCAA audits and other services are required by laws and regulations as outlined in the GAO draft report. DCAA already has a risk-based contract audit approach in identifying resource requirements. DCAA performs annual audit requirements planning procedures to establish staffing requirements based on its regulatory/statutory audit requirements and audit risk. An example of DCAA's risk-based audit approach for incurred cost audit requirements is DCAA's performance of desk reviews of the low risk contractor's incurred costs on a cyclical basis in lieu of performing a full incurred cost audit every year. Nevertheless, USD(C) will task DCAA to coordinate with the Under Secretary of Defense for Acquisition, Technology, and Logistics to assess DCAA audit requirements since any significant change in DCAA audit requirements would involve a change in acquisition policy. In addition, DCAA has engaged the Naval Post Graduate School to assist in its "cultural transformation." One of the projects is identifying and resolving differing stakeholder expectations while ensuring DCAA performs quality audits that meet GAGAS. DCAA's efforts will include consulting with DoD stakeholders and engaging outside experts as part of this project. The Department will complete its assessment by December 2010. Recommendation 6: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to establish an SES-level position with responsibility for audit quality assurance that requires demonstrated knowledge and experience in applying professional audit standards. DOD Response: Concur. In August 2008, DCAA established the position of Assistant Director, Integrity and Quality Assurance with responsibility for DCAA's quality assurance program. On July 9, 2009, this position was approved as an SES-level position and the job announcement has been published. DCAA expects to fill this position by October 2009. Recommendation 7: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to establish a separate DCAA internal review organization to conduct critical internal inspector general functions, including performing periodic internal evaluations and reviews and addressing DCAA hotline complaints. DOD Response: Concur. DCAA will establish a separate internal review organization to conduct critical internal inspector general functions, including performing periodic internal evaluations and reviews and addressing DCAA hotline complaints and serve as the Agency's ombudsman. The new organization will be established by December 2009. Recommendation 8: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to review DCAA's current portfolio of audit and nonaudit services to determine if any should be transferred, or reassigned to another DoD agency, or terminated in order for DCAA to comply with GAGAS integrity, objectivity, and independence requirements. DOD Response: Concur. USD(C) will require DCAA to assess its current portfolio of audit and nonaudit services to determine if any should be transferred, or reassigned to another DoD agency, or terminated in order for DCAA to comply with GAGAS integrity, objectivity, and independence requirements. DCAA will report the results of its assessment to USD(C) by March 2010. However, as stated above, the majority of DCAA audits and non-audit services are required by regulation or statute. As acknowledged by the GAO in its report, DCAA has already performed some of this assessment resulting in the elimination of DCAA auditor participation as members of Integrated Product Teams and Source Selection Evaluation Boards. Recommendation 9: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA based on the risk-based audit approach to develop a staffing plan that identifies auditor resource requirements as well as auditor skill levels and training needs. DOD Response: Concur. DCAA is entering into an agreement with the Army Management Support Agency to evaluate DCAA's requirements planning process and resulting staffing assessment for FY 2010. The Army will also provide an overall opinion on the sufficiency of DCAA staffing to provide audit coverage in line with its mission and strategic plan and fully comply with Generally Accepted Government Auditing Standards. In addition, DCAA has an internal education specialist to provide expert advice on DCAA training and an internal statistician to provide expert advice on statistical sampling audit techniques and training. Recommendation 10: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to establish a position for an expert on auditing standards or consult with an outside expert on auditing standards to assist in revising contract audit policy, providing guidance on sampling and testing, and developing training on professional auditing standards. DOD Response: Concur. The USD(C) will require DCAA to consult with an outside expert on auditing standards to assist in revising contract audit policy, providing guidance on sampling and testing, and developing training on professional auditing standards. Recommendation 11: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to revise DCAA audit policy to provide appropriate guidance on what constitutes sufficient testing to comply with GAGAS. Update DCAA's Contract Audit Manual, as appropriate. DOD Response: Concur. USD(C) will require DCAA to issue audit policy on what constitutes sufficient testing to comply with GAGAS and update DCAA's Contract Audit Manual. DCAA will complete this action by December 2009. Recommendation 12: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to develop agency wide training on government audit standards. This training should emphasize the level of assurance intended by the various types of engagements and provide detailed guidance on auditor independence, planning, fraud risk, level of testing, supervision, auditor judgment, audit documentation, and reporting. DOD Response: Concur. DCAA is currently developing a revised training course on Generally Accepted Government Auditing Standards. This course will emphasize the level of assurance intended by the various types of engagements and provide guidance on auditor independence, planning, fraud risk, level of testing, supervision, auditor judgment, audit documentation, and reporting. All DCAA employees have been notified that they will be required to take this course in FY 2010. DCAA is also increasing the level and complexity of GAGAS training in all courses. Recommendation 13: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to conduct a comprehensive, independent review of DCAA's revised audit quality assurance function. This review should focus on the consistent application of criteria used for assessing audit quality and assuring timely, consistent and appropriate reporting of review results. DOD Response: Concur. On July 30, 2009, DCAA requested the DoDIG to perform its external quality review of DCAA's Quality Control System for audits and attestation engagements for the period ending September 30, 2009. Recommendation 14: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to make appropriate recommendations to address annual quality assurance review findings of serious deficiencies and GAGAS noncompliance, provide training, and follow-up to assure that appropriate corrective actions have been taken. DOD Response: Concur. The DCAA Integrity and Quality Assurance organization has already established a process for making recommendations to address annual quality assurance review findings of serious deficiencies and GAGAS noncompliance to the Director DCAA on a real-time basis. Based on these recommendations, DCAA will ensure that any necessary training is provided and appropriate corrective actions have been taken. Recommendation 15: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense (Comptroller/CFO) to require the Director of the DCAA to establish policies and procedures to ensure that auditors who make direct bill decisions are independent of management employees who review vouchers of contractors not eligible for the direct bill program, thereby reducing situations where DCAA auditors are encouraged to reduce their workload by approving contractors for the direct bill program. DOD Response: Nonconcur. The Department believes that a review of the contractor's interim public vouchers is an integral function of DCAA's continual assessment of a contractor's billing system. A DCAA auditor who audits the contractor's billing system (and potential subsequent approval for direct bill) is in the best position to review and approve contract interim billings based on their thorough understanding of the contractor's system. In regards to the GAO concern that DCAA auditors may not report findings to reduce their workload by approving contractors for the direct bill program, the Department believes the GAO's concerns are mitigated based on the comprehensive supervisory and audit manager reviews. DCAA does not believe that the approval of interim vouchers along with the approval for contractors to be on direct bill results in a lack of auditor objectivity (see Enclosure 4, Comment 8). It should also be noted that since July 2008, the number of contractors on the direct bill program have been reduced by over 200 contractors and segments of many of the largest defense contractors have been removed from the direct bill program in FY 2009. In October 2008, DCAA issued clarifying guidance on contractor eligibility to participate in the direct bill program. This memorandum includes guidance on ensuring sufficient testing is performed when determining a contractor is eligible for direct bill. [End of Enclosure 1] Enclosure 2: GAO Draft Report Dated July 31, 2009 GAO-09-468 (GAO Code 195099) "DCAA Audits: Widespread Problems With Audit Quality Require Significant Reform" Department Of Defense Comments To The GAO Recommendations For Congressional Consideration Recommendation 1: In the short term, as DCAA makes progress in correcting fundamental weaknesses that have impacted audit quality, Congress could consider enhancing DCAA reform efforts by enacting legislation to grant it protections and authorities similar to those embodied in the Inspector General Act, as amended. DOD Response: Nonconcur. DoD generally opposes providing DCAA with authorities similar to those contained in the Inspector General Act. DoD specifically opposes certain recommendations based on the IG model. We do not believe that the DCAA Director should be a Senate confirmed position unless DCAA is independent of DoD. Presidential nomination and Senate confirmation injects a political element into DCAA that is not appropriate and inevitably creates lengthy periods when there is no Director. We also oppose fixed terms for the DCAA Director. If DCAA remains part of DoD, the Secretary of Defense must have the ability to choose an appropriate Director. We also question the wisdom of an independent budget (which would prevent or limit our ability to move money into DCAA, as is occurring now based on funding from the Defense Acquisition Workforce Development Fund). Nor do we support mandatory public reporting, an additional burden on an agency that is already struggling to meet its many mission demands. While we do not support independence based on the IG model, we plan to take steps to strengthen DCAA's independence by establishing an appeals process that permits DCAA to seek resolution when there are differences of opinion as to the resolution of its audit findings. Under this process, DCAA could appeal differences first to the Director of Defense Procurement and Acquisition Policy (DPAP). If DCAA disagreed with the DPAP decision, DCAA would be permitted to appeal to the Under Secretaries of Acquisition, Technology, and Logistics and Comptroller, acting as a team. We expect that appeals at the Under Secretary level would be rare and would involve only the most important issues. Recommendation 2: In the medium term, Congress could consider elevating the contract audit function within DOD by moving DCAA from under the DOD Comptroller/CFO and placing it under the Deputy Secretary of Defense. DOD Response: Nonconcur. DoD strongly opposes this recommendation. The Deputy Secretary is the Chief Management Officer of one of the world's largest organizations and backs up the Secretary in the wartime chain of command. The Deputy simply does not have the time to provide oversight and support to individual defense agencies. [End of Enclosure 2] Enclosure 3: DCAA Actions Taken Since the Issuance of GAO Report GAO-08-857: Structure: * Approved Agency-wide reduction in supervisory span of control (June 2008). * Approved 25 new field audit offices and 5 new Regional Audit Managers lowering the span of control (May - February 2009). * Completed Agency-wide staffing assessment and requested staffing increase to Comptroller on September 10, 2008. Updates on staffing shortfalls were provided to the Comptroller at regular intervals throughout FY 2009. * Realigned Quality Assurance to report directly to the Deputy Director (August 2008). - Submitted request to OSD for SES level position for the Integrity and Quality Assurance (QA) function (September 2008). Request was initially denied by DoD in January 2009 and the position was filled at the GS-15 level. However, after another attempt by the Director for a SES position, DCAA received approval in July 2009 and a job announcement was issued shortly thereafter. - Expanded the next round of QA reviews. - Revised process for tracking and following-up on QA findings. - Revised process for next 3-year cycle to ensure all audit offices are covered, after consultation with the DoD IG. - Completed assessment on level of QA staffing. - Issued revised comprehensive instruction on DCAA's QA program (December 2008). * Submitted request for funds under Section 852 acquisition workforce fund in December 2008. Under the Defense Acquisition Workforce Development Fund, DCAA has received $17.2 million to date (allotments in March, April, and August). * DCAA brought on-board 245 new interns by the end of July and have many offers with on-board dates in late FY 2009. As a result, DCAA will easily meet the goal of 300 by the end of September and will probably exceed it. * Realigned all Financial Liaison Advisors from the Field Detachment region (region that handles all Top Secret audits) to Headquarters to avoid the appearance of a lack of independence. As of November 2008, all Financial Liaison Advisors report directly to Headquarters. * At the request of the Director, the DCAA point of contact for the Office of Special Counsel investigation was moved from the DCAA General Counsel to the DoD General Counsel's office due to the investigation being expanded. Culture: * Revised policy for resolving differences in audit results and opinions - elevate within management structure from two to four levels (July 2008). * Ceased participation as members of Integrated Product Teams (IPTs) to avoid the appearance of a lack of independence (August 2008). * Revised performance measures - eliminated 18 measures and added 8 measures (September 2008). * Established an anonymous website for employees to voice concerns with the inappropriate use of performance measures and other inappropriate actions (September 2008). * Engaged OPM to conduct an organizational assessment survey and are assessing results of the survey conducted by OPM - the working group is evaluating results and developing actions (assessment due August 2009). * Ceased participation as members of Source Selection Evaluation Boards to avoid the appearance of a lack of independence - requested audits will still be provided (November 2008). * Director/Deputy Director staff presentations emphasize the need to perform quality audits and discuss performance measures (various presentations through 2008 and 2009). * Established a Senior Advisory Council for Improvement chaired by the Director to Oversee the implementation of improvements as a result of the Defense Business Board recommendations (report issued January 22, 2009). * Issued several memorandums reiterating the importance of cooperating with GAO, IG and other reviewers/investigators. * Held stand down day for audit quality at all DCAA locations (August /September 2008 and again in August 2009). * Completed annual independence training (September 2008). * Held focus groups to obtain feedback on implementation of performance measures issued in September 2008 which revealed minimal problems with implementation of new measures (February/March 2009). * The Director required all regions to assess whether exceeding budget hours on individual assignments was inappropriately used to lower performance ratings. The regions completed the assessments and, where needed, have implemented corrective actions (December 2008). * Established new process to obtain input regarding the new hire employment experience and to identify reasons why employees leave DCAA (November 2008). * Revised job objectives/performance plans for the 0511 (auditor) positions to eliminate the language on meeting audit budget hours and productivity measures and added language strengthening the need to execute audits in accordance with the auditing standards and Agency policy (February 2009). * Revised supervisory development curriculum based on feedback from focus groups and other feedback mechanisms to emphasize leadership skills and the more common day-today activities which supervisors perform (April 2009). Processes: * Issued memorandum on adequate working paper documentation (July 2008). * Completed Agency-wide assessment to determine whether GAO's findings are systemic across DCAA. Six of the forty assignments reviewed contained noncompliances. Actions being taken to address issues (September 2008). * Raised the field audit office signature authority for all audit reports to the level of the manager or higher (August 2008). * Revised policy for the monthly quality review of issued audit reports from regions to the Headquarters Quality Assurance division (October 2008). * Revised DCAA Quality Checklist for Review of Audit Working Papers (checklist is used by auditors and supervisors prior to report issuance) (December 2008). * Issued guidance clarifying DCAA's process for pursuing access to contractor records and initiating a subpoena (December 2008). * Issued clarifying guidance on what constitutes a significant deficiency in contractor internal control systems (December 2008). * Revised policy on reporting results of the review of contractor systems and related internal controls to eliminate the inadequate in- part opinion so that the overall opinion on the system is either adequate or inadequate (December 2008). * Issued guidance on performing and reporting on limited scope internal control audits (December 2008). * Issued guidance reminding auditors to report suspected contractor fraud and other irregularities encountered during the audit and emphasized that managers do not approve the Form 2000, but rather review it for clarity (February 2009). * Issued guidance on documentation of judgmental sampling (February 2009). * Revised guidance for reporting unsatisfactory conditions related to actions of Government officials wherein certain unsatisfactory conditions will be reported directly to the DoDIG in lieu of reporting the conditions to a higher level of management (March 2009). * Issued guidance clarifying requirements for contractor eligibility to participate in the direct bill program (April 2009). * Issued guidance to remove major contractors from direct billing where contractor has implemented a new billing system or accounting system that significantly impacts Government billings and the new system has not been examined (April 2009). * Revised a self-study training course (CMTL 1326) to include new guidance on identifying key elements of an effective internal control audit report and the requirements for issuing a real-time (flash) report (May 2009). * Issued an audit alert emphasizing existing guidance which requires that a separate cost accounting standards (CAS) noncompliance audit report will be issued when a CAS noncompliance is found during any audit (June 2009). * Issued an audit alert to clarify that forward pricing due dates should be based on the realistic assessment of risk factors for each specific contractor and proposal under review (June 2009). * Issued guidance on contract audit closing statement reviews in July (after receipt of DoD IG comments). This completes the last action item from the peer review. Long-Term Planned Actions: * Obtained the services of the Naval Postgraduate School, Center for Defense Management Reform to assist with the Agency-wide cultural transformation. The initial effort started June 2"d with the DCAA executive team. As a result, four major initiatives were adopted for incorporation in the DCAA Strategic Plan. Teams of executives were assigned to each initiative to further develop the milestone plan for executing the objective. The four items are: 1. How can DCAA put people first to guide its decisions, actions and values? For example, how can DCAA place an increased emphasis on "soft skills" such as building morale and developing employees (in terms of a broad understanding as well as technical proficiency)? 2. How can DCAA develop leaders to serve the employees and the organization? 3. How can DCAA structure the organization to facilitate compliance with GAGAS, maximize audit results/ROI, and better align Agency workload/resources? 4. How can DCAA identify and resolve differing stakeholder expectations with contracting officers, contractors, the public (Congress), and external review organizations? These items will be worked for about the next three years. Once the milestone plan for each of the four initiatives is developed, it is envisioned that each objective will have various completed actions throughout the next three years. Once the milestone plans are developed, the objectives will be communicated to the workforce. * Performing a comprehensive assessment and revision to DCAA training by instituting a life-cycle training process. Effort started in FY 2008 and will conclude in about three years. * Conducting a comprehensive organizational assessment (based on Baldrige). Estimated completion in FY 2010. * Performing a comprehensive review of DCAA's approach for performing internal control audits. Estimated completion of baseline audit opinions in FY 2010. * Engaging the Army Force Management Support Agency to evaluate DCAA's process for planning FY 2010 audit needs as well as staffing requirements. The effort is expected to be completed by the end of September. [End of Enclosure 3] Enclosure 4: GAO Draft Report Dated July 31, 2009: GAO-09-468 (GAO Code 195099): "DCAA Audits: Widespread Problems With Audit Quality Require Significant Reform" Department Of Defense Comments To The GAO Report Narrative: 1. GAO Narrative: Nationwide audit quality problems are rooted in DCAA's poor management environment (page 11). DOD Comments: GAO stated about half of the rescinded reports relate to unsupported opinions on contractor internal controls. This is incorrect. The majority of the rescinded reports relate to forward pricing reports based on the prior GAO investigation report issued in 2008. [See comment 1] 2. GAO Narrative: Audit Quality Problems Found in All Audits GAO Reviewed, Independence Issues (page 12). DOD Comments: GAO stated that in 8 audits they reviewed, DCAA independence was compromised because auditors provided material nonaudit services to a contractor they later audited; experienced access to records problems that were not fully resolved; or significantly delayed report issuance in order to allow the contractors to resolve cited deficiencies. The Department understands the GAO's concerns, however, we believe the auditors' intent of providing preliminary audit results and discussing draft policies and procedures was generally an attempt to ensure the evidence was accurate and to expedite contractor actions so that contractor systems would be corrected promptly to minimize the risk of overpayments. DCAA acknowledges that significant time lapsed in several of these assignments between the time the contractor was provided the draft findings and the issuance of the final audit report. DCAA concurs that auditors should not provide input to contractors on draft policies and procedures and that the reports should identify all deficiencies found even though they have been corrected at the time of report issuance. In memorandums issued in August and September 2008, DCAA issued guidance prohibiting auditors from providing input to contractors on such items as draft proposals, draft policies and procedures, and draft disclosure statements and to require auditors to report all deficiencies found even when the contractor corrects the deficiencies during the audit. 3. GAO Narrative: Audit Quality Problems Found in All Audits GAO Reviewed, Insufficient Evidence (page 12). DOD Comments: GAO stated that 33 of the 37 internal control audits did not include sufficient testing to support auditor conclusions and opinions. GAO stated that GAGAS for examination-level attestation engagements require that sufficient evidence be obtained to provide a reasonable basis for the conclusion that is expressed in the report. We agree with GAO that in several cases the testing was insufficient. The level of testing was based on the auditor's judgment and the total audit concept - drawing on an accumulation of knowledge and experience previously gained at the contractor based on auditing that contractor on a continual basis. Nevertheless, DCAA acknowledges the testing was insufficient to support an internal control audit opinion, and therefore, has rescinded several of these reports, increased the level of testing on related audits and commenced new full scope internal control audits at these locations. 4. GAO Narrative: Table 2, Summary of Five Selected Internal Control Audits, Eastern Region - Billing System Audit (page 16). DOD Comments: GAO stated that the subject audit had an impairment to auditor independence because the auditors helped the contractor develop policies and procedures. We generally concur with the GAO in that the FAO was performing IPT type effort in providing the contractor comments on draft policies and procedures. DCAA believes the intent of these actions was to expedite contractor corrective action so that the contractor's system would be corrected to minimize the risk of overpayments and protect the taxpayer's interests. DCAA recognized the potential appearance of a lack of independence related to this kind of effort and on August 11, 2008, issued audit guidance that prohibited auditors from providing input to contractors on such items as draft policies and procedures regardless of the circumstances. 5. GAO Narrative: Table 2, Summary of Five Selected Internal Control Audits, Central Region - Billing System Audit (page 16). DOD Comments: GAO stated that the subject audit had an impairment to auditor independence because the auditors monitored contractor corrective action for 7 months instead of issuing the audit report when the work was completed. DCAA concurs with the GAO that the significant lapse of time from the date the draft results were provided to the contractor and the date the audit report was issued gives the appearance of a lack of independence. However, the field audit office does not believe it intentionally delayed the issuance of the report to allow the contractor to correct the system. Staffing issues and an increase in higher risk assignments played a significant role in the delay. [See comment 2] 6. GAO Narrative: Table 2, Summary of Five Selected Internal Control Audits, Central Region - Billing System Audit (page 16/17). DOD Comments: The GAO narrative appears to imply that DCAA was not aggressive in pursuing the potential fraud at this contractor. It should be noted that the DCAA Investigative Support Team was instrumental in assisting in this fraud investigation that resulted in recovering more than $2.8 million dollars in a civil case settlement. [See comment 3] 7. GAO Narrative: Cost Related Assignments (page 17). DOD Comments: GAO stated that the 32 cost-related assignments did not contain sufficient testing to provide reasonable assurance that overpayments and billing errors that might have occurred were identified. It appears that the GAO is holding DCAA to the GAGAS requirements for these assignments even though the majority is not GAGAS-type audits based on Agency policy. DCAA does not agree that all of the 32 cost-related assignments contained insufficient testing, however, DCAA is continuing to assess the GAO's comments and will develop an appropriate action plan. [See comment 4] 8. GAO Narrative: Auditor objectivity issues (page 20). DOD Comments: GAO stated that DCAA's role in approving contractors for participation in the direct bill program presented an impairment to auditor objectivity- which includes being independent in fact and appearance when providing audit and attestation engagements. The GAO noted that GAGAS state that audit organizations should not authorize an entity's transactions or audit their own work. GAO further stated that DCAA's role in authorizing contractors to participate in the direct bill program places it in the position of making decisions that impact its own workload related to review of contractor invoices prior to payment. Although not explicitly stated, the GAO seems to imply that when DCAA performs an audit of the contractor's billing system (and potentially approves the contractor for the direct bill program) while also having the responsibility for approving interim vouchers, DCAA is authorizing the contractor's transactions and it is auditing its own work. In addition, the GAO seems to imply that DCAA's objectivity is impaired and DCAA auditors may overlook findings to reduce their workload by approving contractors for the direct bill program. DCAA disagrees with both implications. [See page 76] DCAA is not authorizing a contractor's transaction or auditing its own work either when it approves the contractor for direct bill or approves an interim voucher for payment. GAGAS 3.14 (GAO-03-673G) defines authorizing an entity's transactions as an example of performing management functions or making management decisions. When DCAA approves a contractor's voucher for payment it is not acting on behalf of the contractor (i.e., performing a contractor management function) but acting as a representative of the Government in its role as an independent external oversight organization based on the authority provided for in DFARS 242.803. The contractor's interim vouchers submitted to the Government for payment are prepared by contractor employees and authorized by contractor management. DCAA also does not believe that performing the function of an external oversight organization by approving contractor's interim vouchers or approving the contractor for the direct bill program places DCAA in the position of auditing its own work when it later audits the contractor's billing system. These functions do not involve developing the contractor's billing system policies and procedures and internal controls. In addition, as stated above, DCAA also is not involved in preparing the contractor's vouchers. Therefore, approving contractor's interim vouchers or approving the contractor for the direct bill program and subsequently auditing the contractor's billing system does not result in DCAA auditing its own work. Furthermore, DCAA believes that its role in reviewing and approving interim vouchers falls within the category of nonaudit services discussed in GAGAS A3.02 and A3.03 (GAO-07-731G) that do not impair independence. We also disagree that DCAA's role in authorizing contractors to participate in the direct bill program results in an impairment to independence because performing that function places DCAA in the position of making decisions that impact its own workload. In our opinion the GAGAS independence standards do not preclude the auditor from performing such functions. In fact, conclusions reached in many of the audits and other functions typically performed by auditors have the potential to impact the audit organization's workload. For example, a qualified or adverse opinion on the company's financial statements could prompt the company to replace the independent public accountant (IPA) that performed the audit; thereby negatively impacting the IPA's future workload and revenue. The GAO's concerns are mitigated by the fact that DCAA supervisors are actively involved in all aspects of the audit including review and approval of the risk assessment, interim reviews as needed, and a final comprehensive supervisory review of the working papers and draft report. Technical specialists may also review the risk assessment and audit conclusion. In addition, field audit office managers review sensitive and high risk audits and may elevate them for regional management review prior to report issuance. GAO stated that DCAA had approved direct billing on all but 2 of the 16 contractors they reviewed. GAO reviewed 20 billing system audits covering 17 contractors, only 2 of those contractors are still on direct billing. It should also be noted that since July 2008, the number of contractors on the direct bill program have been reduced by over 200 contractors. 9. GAO Narrative: DCAA's audit quality assurance program was ineffective (page 25). DOD Comments: GAO stated that DCAA's audit quality assurance program was not properly implemented, resulting in an ineffective quality control process that accepted audits with significant deficiencies and noncompliance with GAGAS and DCAA policy. We do not agree with the statement that the quality assurance process accepted audits with significant deficiencies. As the GAO states in its report, the quality assurance reviews did identify significant GAGAS noncompliances which they reported to the field audit office for corrective action - they did not "accept" these audits. DCAA has recently made several improvements to the quality assurance program to include increasing the level of management responsible for developing and implementing a corrective action plan. The current program requires the Regional Director to ensure corrective actions have been taken and all corrective actions will be tested by the Headquarters quality assurance staff to ensure the noncompliances have been corrected. [See page 78] 10. GAO Narrative: DCAA lacks a risk-based audit planning approach (page 28). DOD Comments: GAO stated that DCAA lacks a risk-based audit planning approach. We do not agree with this statement. DCAA has a risk-based contract audit approach in identifying resource requirements. DCAA performs annual audit requirements planning procedures to establish staffing requirements based on its regulatory/statutory audit requirements and audit risk. [See page 75] 11. GAO Narrative: Allegations about abusive management actions have continued (page 32). DOD Comments: The GAO states that allegations about abusive management actions have continued. The report further states that "DCAA Headquarters officials explained that in several cases, Western Region management has not agreed to take disciplinary or other available corrective action. The officials told us that DCAA hotline staff has no recourse in these situations." The DCAA hotline group functions as a finder of fact, and they recommend corrective actions. While it is true that the DCAA hotline group has no authority to take disciplinary action itself or to require corrective action, any disagreement between the hotline team and the region may be resolved by the DCAA Director or Deputy Director. The Western Region management has agreed with the recommended actions provided by the hotline team in a final formal report. Several cases are still in the investigative stage and although interim recommendations may not have been adopted yet by the region, we fully expect the Western Region management to support the recommendations made in a final report. 12. GAO Narrative: Increase authority and independence (Page 50). DOD Comments: GAO stated that legislation could strengthen DCAA's audit authority by providing the same level of access to records and personnel available to IGs. GAO goes on to state that currently, DCAA has access to certain records related to cost-type contracts or that contain cost and pricing data, but not to contractor personnel. We do not agree with the statement that DCAA does not have access to contractor personnel. DCAA interfaces with contractor employees during all phases of the audit, which include interviews, inquiry, observation as well as providing draft audit results for comment and verification of factual content. [See comment 5] 13. GAO Narrative: Independence Impairments, (Pages 60-61). DOD Comments: The GAO provides an example in this narrative section where the GAO contends that the field audit office experienced an access to records issue. The GAO concludes that since the contractor inquired regarding the auditor's need for certain data, there was an access to records issue. DCAA routinely encounters "push back" from contractors, this does not constitute an access to records impairment if the auditor does not succumb to the pressure and obtains the requested data. The field audit office does not believe its access to data relating to the training records was denied by the contractor as cited by the GAO. The contractor subsequently provided the requested data to the auditor. [See comment 6] 14. GAO Narrative: Failure to Design and Perform Procedures to Detect Fraud Risk (Pages 64-66). DOD Comments: The GAO provides an example in this narrative section where the GAO contends the FAO manager and supervisor ordered an auditor to ignore significant fraud risks during the audit. DCAA does not agree with this conclusion and has not been provided evidence to support this assertion. It should be noted that the DCAA was instrumental in assisting in this fraud investigation that resulted in recovering more than $2.8 million dollars in a civil case settlement. [See comment 3] Defense Procurement and Acquisition Policy (DPAP): Collateral Action Officer's Comments on GAO Draft Report: "DCAA Audits: Widespread Problems With Audit Quality Require Significant Reform" Following are the DPAP's Collateral Action Officer's (CAO's) technical comments regarding the Government Accountability Office's (GAO's) July 31, 2009 Draft Report GAO-09-468 (GAO Code 195099) "DCAA Audits: Widespread Problems With Audit Quality Require Significant Reform". 1. Auditing in the Public Interest: The report, in part, impugns DCAA's audits stating, "DCAA's management environment ... put DCAA in the role of facilitating DoD contracting without also protecting the public interest."[Footnote 1] Likewise, the GAO mentions that the "DCAA mission should be refocused to protect the taxpayer interest ..." compared to the current mission that "fostered the culture of supporting contracting officials[Footnote 2] 2 As a result, the GAO agrees with the Defense Business Board's recommendation to "revise DCAA's mission to focus on protecting the interest of taxpayers, with the taxpayer as the primary customer...[Footnote 3] [See page 78] The report adopts a position that because DCAA is serving the interests of Contracting Officers, that DCAA is not auditing in the interest of the public. DCAA is a service organization created to provide financial information, audits, and advice to support decision making by DoD Contracting Officers. DCAA serves the public interest by providing timely and useful information to Contracting Officers. It is erroneous to imply that contracting officers do not seek to protect the public interest. The contracting officer is bound by regulation to meet the public interest in the broadest sense, for the entire matter surrounding a contract. The contracting officer, in the award and administration of a contract, is the government official responsible for insuring that all requirements of law, executive orders, regulations, and all other applicable procedures, including clearances and approvals, have been met. Any logic that presumes that by focusing on supporting contracting officials DCAA somehow failed to act in the public interest is flawed in our view. Someone trained and named in both law and regulation has to look at the larger picture, and not just the audit, if the public interest is to be served-the contracting officer is that person. Numerous provisions of the Federal Acquisition Regulations (FAR), for instance, speak to the issue of all members of the acquisition and administration community serving the public interest: An essential consideration in every aspect of the System is maintaining the public's trust. Not only must the System have integrity, but the actions of each member of the Team must reflect integrity, fairness, and openness. The foundation of integrity within the System is a competent, experienced, and well-trained, professional workforce. Accordingly, each member of the Team is responsible and accountable for the wise use of public resources as well as acting in a manner which maintains the public's trust. Fairness and openness require open communication among team members, internal and external customers, and the public.[Footnote 4] Specifically to contracting officers, FAR states: Contracting officers are responsible for ensuring performance of all necessary actions for effective contracting, ensuring compliance with the terms of the contract, and safeguarding the interests of the United States in its contractual relationships. In order to perform these responsibilities, contracting officers should be allowed wide latitude to exercise business judgment.[Footnote 5] By serving the interests of Contracting Officers well, DCAA does serve the public interest. It is not one or the other as might be interpreted in the report. 2. Production Auditing: Throughout the report, the GAO implies and in some instances states that the problem is "Production-Oriented Auditing" and that audits have been rushed by contracting officer requirements. This sets up a dichotomy between "quality" audits and timely audits. We strongly urge the GAO to consider that an audit not delivered in time to be useful, is of limited value to the Government. [See comment 7] For example, a proposal review delivered after negotiation has started and decisions have already been made is of greatly diminished usefulness. Likewise, an incurred cost review that is not completed in a reasonable period after the costs are incurred loses contemporaneous support-employees of the contractor and the Government leave, some records are lost or are placed in deep storage. Similarly, business system reviews need to be issued while a problem is still subject to correction before the fact to protect the Government's interests, not two and three years later. Timeliness is a critical element of quality. Delays in award have consequences to the warfighter. Contracting officers are required to consider those consequences and hence, they are very concerned that DCAA has not been able to consistently deliver timely reports and advice. The GAO has previously recognized there are consequences to award delays:[Footnote 6] "Although federal regulations do not specify how long the contract award process should take, the regulations state that the purpose of planning the contract award process is to ensure that the government obtains the needed goods or services in the most effective, economical, and timely manner. Therefore, written plans for carrying out the award process must include milestones for completing the steps in the process, such as when the agency plans to solicit and evaluate proposals and make the award. Developing and adhering to these schedules can help ensure that the department conducts the process efficiently and can help companies make informed business decisions regarding the allocation of their resources and whether to compete for a contract." "Delays in awarding contracts could increase costs... and could also affect the willingness of companies to compete for future...contracts ....Specifically, in addition to investing time and resources in developing proposals, once a company submits a proposal ... the company is generally required to ensure that the key personnel identified in the proposal continue to be available until the decision is made and the contract awarded....Increased costs and the length of time it takes ...to award a contract also have the potential to affect competition for future...work." GAO's recognition of this fact in the aforementioned report, while important, is still a comparatively limited view of timeliness compared to a contracting officer's point of view. A contracting officer knows that delays can impact funding decisions and disrupt program management plans. Contracts are often interrelated and codependent, such that a delay in awarding one contract can delay an entire system and put off fielding dates, with consequences distributed and cascading through a range of other contracts and plans, and might ultimately result in mission failure. The contracting officer, also by regulation, has to consider and respect the opinions of other specialists that he works with, and not just that of the auditor. A good audit in time is better than an extraordinary audit that is late and never used. An audit is a tool the contracting officer uses to negotiate and administer a contract, but in order to realize the benefits of the auditor's work, the audit must be timely. 3. Generally Accepted Government Auditing Standards (GAGAS): As the GAO correctly points out, DCAA performs most of its audits and reviews in conformance with GAGAS. We believe that for some reviews and financial advice provided by DCAA, it is possible that it may not be necessary to provide a fully conforming GAGAS audit report to support certain contracting officer functions (e.g., providing an attestation review rather than a financial audit). DCAA should use auditing standards and techniques that produce creditable information that can be relied upon by the contracting community in the awarding and administrating of contracts. However, we believe that all types of DCAA reports and reviews should be examined to determine if the standard being applied and reported by DCAA is the appropriate standard given the requirement causing the audit or review to be performed. [See comment 8] 4. Staffing: While the report cites examples of poor quality audits and some poor decisions made by DCAA management, most would seem to be heavily influenced by lack of adequate staffing. Based on our discussions with contracting officers, contractors, and auditors, some and possibly most of the reductions in audit scope and responsiveness by DCAA is a direct result of the staffing drawdown while workload increased. Until the staffing issues are resolved, it will not be possible for DCAA to perform at the level of quality and efficiency that is desired. Rebuilding the DCAA workforce, while a challenge, can and must be done. The workforce build-up will require years of effort to hire and train the staff required to do the work envisioned by the GAO audit. [See comment 9] 5. Placement of DCAA in the Executive Branch: The report raises the question of DCAA's placement within DoD or the Executive Branch. Currently DCAA performs most of the contract auditing functions within the Executive Branch. Even if DCAA performed the remaining contract audits it currently is not performing, DoD would remain by a large factor the majority user of the DCAA services. We do not believe that any useful purpose would be served by moving DCAA outside DoD. DoD has the most vested interests in a well functioning DCAA. [See page 74] In our view, in reporting to the Comptroller, DCAA is insulated from direct influence from contract procurement and contract administration offices. For similar reasons DCMA is in the AT&L chain of command to insulate it from pressures it might have placed on it, if it were in the same chain of command as the procurement offices. The Comptroller is in the best position to understand the DCAA requirements while maintaining its independence from the audit report users. 6. Risk Based Auditing: According to the DCAA Contract Audit Manual,[Footnote 7] all audit planning is risk based. This applies to both the annual planning for types of audits and staff requirements as well as for the planning of specific audits. The Audit Manual is quite clear that the final budget set for the assignment is to be based on the circumstances and risk attached to the assignment being planned. Further, it also clearly sets out that as circumstances change or the risk is found to be different than considered during the planning stage, that budget changes should be made. [See page 75] We are faced in the GAO report with the finding that budgeted hours do not reflect the risk and that changed risk found during the field work has not resulted in changed budgets. This is obviously not a policy matter of DCAA not doing risk-based auditing, since the findings are clearly at odds with the DCAA policy. We believe that the failure to follow the policy is a result of staffing constraints that made it impossible for DCAA to perform all the assigned review requirements to the standards expected. 7. Direct Billing: This GAO draft report seems to misunderstand the Direct Billing program and the problems noted in DCAA administration of voucher reviews. Direct Billing approval was not designed to reduce review of vouchers. It was designed to administratively take advantage of technology to better process vouchers in an efficient manner and to better comply with the Prompt Payment Act. In appropriate circumstances using risk- based analysis of contractor past performance and the quality of its business systems, contractors were to be allowed to be paid before review of vouchers instead of requiring review of the vouchers before payment. [See page 76] The nature of the review program for any given contractor should not have changed due to placement on Direct Billing. The DCAA guidance requires voucher reviews of all contractors every year that the contractor has Direct Billing authority. Where DCAA believed based on past performance or poor systems that there was a significant chance of improper billing, the contractor was not to be included in the Direct Billing program. The review program for the contractors should have been the same as it would have been based on risk factors even if there was no Direct Billing program. The problem with voucher reviews both before the Direct Billing program and after the initiation of the program is that DCAA did not have sufficient staff to perform the reviews required by the risk-based analysis. New contractors and problem contractors should have voucher reviews before payment just as required by DCAA policy. Established contractors with adequate past performance should have vouchers reviewed after payment using a reasonable plan tailored to the contractor's circumstances just as required by DCAA guidance. Changing the decision authority for participation in Direct Billing should have no impact on what vouchers are reviewed. Taking a contractor off of the program does not ensure that the vouchers will be properly reviewed prior to payment if there is not sufficient staff to perform the reviews. Footnotes for Appendix IV: [1] Executive Summary. [2] Page 34. [3] IBID. [4] FAR 1.102-2 (c)(1) Performance standards. [5] FAR 1.602-2 Responsibilities. [6] GAO-06-722, DOE Contracting, Better Performance Measures and Management Needed to Address Delays in Awarding Contracts, June 30, 2006. [7] Chapter 3, Section 100. The following are GAO's comments on the U.S. Department of Defense letter dated September 4, 2009. GAO Comments: 1. Rescinded reports. DOD stated that we were incorrect in stating that about half of the rescinded reports relate to unsupported internal control reports. DOD is correct. Of the 80 rescinded audit reports, 24 (31 percent) relate to unsupported opinions on contractor internal controls, 47 (61 percent) relate to forward pricing reports, and 6 (8 percent) relate to defective pricing, compliance with cost accounting standards, and a labor floor check. We have corrected this information in the body of our report. 2. Central Region billing system audit. DOD stated that the DCAA field audit office does not believe that it intentionally delayed issuance of the report to allow the contractor to correct the system. Audit documentation clearly shows that the auditors monitored the contractor's actions for 7 months and issued the audit report 9 months after the exit conference, once the contractor had prepared "written desk procedures to ensure liquidation progress billings would be handled correctly." Opinions should be based on the findings at the end of the audit, and reports should be issued when the audit is completed. 3. Central Region billing system audit during fraud investigation. DOD stated that our report appears to imply that DCAA was not aggressive in pursuing the potential fraud at this contractor and noted that the DCAA Investigative Support Team was instrumental in assisting in the fraud investigation that recovered over $2.8 million in a civil case settlement. The audit documentation shows that the Regional Audit Manager, in the presence of the field office manager and the supervisory auditor, directed the auditor not to pursue contractor charges of costs to future-year, unfunded contract lines and to forget what she learned in her previous DOD contract administration job where she had been responsible for reviewing similar types of contracts. In addition, after reassignment of the first supervisor, the second supervisory auditor instructed the auditor to stop "over documenting" her audit, to complete the assignment, and issue the report. Moreover, the DCAA auditors who investigated the fraud worked with the Army Criminal Investigative Division special agent and Department of Justice Attorneys, not the DCAA field audit office. Finally, we did, in fact, discuss additional documentation we received from the investigative auditors with DCAA headquarters officials and provided them a summary of the key audit-related issues that we obtained from the investigators. 4. Insufficient testing in cost-related assignments. DOD stated that it appears that GAO is holding DCAA to the GAGAS requirements for these assignments even though the majority of these audits are not GAGAS-type audits. As discussed in our report, DCAA does not consider its paid voucher reviews and overpayment assignments to be GAGAS assignments. However, this is important work intended to assure the reliability of contract payments. Specifically, DCAA paid voucher reviews are relied on for making billions of dollars in continuing contract payments without prior review by the government. The Defense Finance and Accounting Service (DFAS) relies on DCAA voucher reviews, and DFAS certifying officers do not repeat review procedures they believe were performed by DCAA. Because paid voucher reviews constitute a payment audit, they require sufficient testing to support reported DCAA conclusions that the government can rely on contractor controls over preparation of interim vouchers to continue to make contract payments without prior review. In addition, DCAA's overpayment audits are intended to determine whether the contractor has adequate controls in place to detect and correct causes of overpayments and billing errors and make timely refunds and adjustments. The limited testing we observed in our work does not provide the intended assurance. 5. Increase authority and independence. DOD stated that it did not agree with our statement that DCAA does not have access to contractor personnel. The discussion in our report is based on DCAA's authority in 10 U.S.C. 2313(a)(B)(2), which gives DCAA legal access to certain contractor records but not access to contractor personnel. Further, DCAA subpoena authority in 10 U.S.C. 2313(b) is specific to the production of contractor records that DCAA is authorized to audit or examine and does not cover contractor personnel. We agree that in practice, DCAA auditors have numerous ongoing discussions with contractor personnel. However, if a contractor official refuses to talk to an auditor, DCAA does not have legal authority to compel contractor officials to meet with or talk to DCAA auditors. Our point is that under authority similar to the IG Act, DCAA's authority to interview contractor officials would be enhanced. 6. Independence impairments. DOD stated that DCAA routinely encounters "push back" from contractors and that the DCAA field office subsequently received training records from the contractor. We recognize that the field office received some training records from the contractor. However, we saw a pattern throughout this audit where the auditor limited requests for contractor documentation and also performed little or no testing in various areas because "the contractor would not appreciate it" if he did more testing. The audit documentation shows that the auditor performed limited testing of selected billing clerk training. Additionally, documentation on testing of the contractor's review of subcontractor costs shows that although the auditor should have tested cost data for three of the top five subcontractors, the auditor asked the contractor to "provide a list of the top 3-5 subcontracts, including subcontract values.... Three subcontractors would be fine." This indicates that the auditor not only accepted data the contractor was willing to provide for testing, but he also let the contractor select the data to be used for testing. The auditor then tested costs of only one subcontractor. The pattern of backing off on requests for documentation and limiting the extent of testing based on concerns about the contractor's reaction indicates that the auditor was influenced by the contractor and limited his audit procedures as a result--a clear independence impairment. DCAA rescinded the audit report on February 10, 2009. 7. Production auditing. USD AT&L comments suggest that there is a trade- off between audit timeliness and audit quality. We view both quality and timeliness as critical to effective contracting officer decision making. However, timely audits that do not meet professional standards are not quality audits and could be misleading or impair important contract decisions. For example, our audit identified three contractor internal control audits--an accounting system audit and two billing system audits--that were completed in 9, 13, and 15 days, respectively-- all with adequate opinions on the contractor's internal controls. Apparently, the contracting officers involved thought these audits were timely and met their needs because there was no audit documentation to the contrary. However, in response to our work, DCAA has rescinded all three of these audits. USD AT&L comments also stated, "A good audit in time is better than an extraordinary audit that is late and never used." Our report did not call for extraordinary audits. DOD has determined that certain DCAA audits should comply with professional standards. When audit organizations state that their audits comply with professional standards, they must follow these standards. Further, until DCAA and AT&L address the need for DCAA to perform 30,000 assignments and issue over 20,000 reports annually, DCAA will continue to face audit quality and timeliness problems. 8. Contract audits in conformance with GAGAS. USD AT&L states that it believes that for some reviews and financial advice provided by DCAA, it is possible that it may not be necessary to perform GAGAS work to support certain contracting officer functions. We agree. As discussed in our report, a risk-based audit approach may require an appropriate delegation of nonaudit contract administration activities and audit responsibilities among DCMA, buying commands, finance community, and DCAA. An effective risk-based approach would include an effort by these communities to re-evaluate whether all such services should be provided as audits and whether DCAA, as an independent audit organization, would perform any nonaudit services. USD AT&L also stated that DCAA may be able to support contracting officer functions through an attestation review rather than a financial audit. However, DCAA does not perform financial audits. Instead, DCAA performs examination-level attestation audits and reports conclusions and opinions on subject matter as a whole. Examinations provide the highest level of assurance, and they must be based on sufficient evidence, often referred to as positive assurance work. For an attestation review, GAGAS require auditors to perform sufficient testing to form a conclusion based on the work performed. It is important to note that GAGAS prohibit auditors from performing review- level attestation work for reporting on internal control or compliance with laws and regulations. 9. DCAA staffing. USD AT&L stated that "the [DCAA] workforce build-up will require years of effort to hire and train the staff required to do the work envisioned by the GAO audit." We did not call for a build-up of the DCAA workforce. Instead, we noted that DCAA production metrics had a direct impact on audit quality. Therefore, it will be important to perform a risk-based analysis of FAR requirements and determine the mix of audit and nonaudit services that will best meet these requirements with consideration of appropriate roles and responsibilities of the contracting and finance communities. [End of section] Appendix V: Comments from the Department of Defense Inspector General: Inspector General: Department Of Defense: 400 Army Navy Drive: Arlington, Virginia 22202-4704: September 3, 2009: Gregory D. Kutz: Managing Director: Forensic Audits and Special Investigations: Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Re: GAO Draft Report, "DCAA Audits: Widespread Problems with Audit Quality Require Significant Reform," dated July 31, 2009 (GAO Code 195099/GAO-09-468): Dear Mr. Kutz: Thank you for the opportunity to respond to the subject draft report. We recognize the critical role that the Defense Contract Audit Agency (DCAA) plays in contractor oversight inside and outside the Department of Defense, and we take very seriously concerns about compliance with generally accepted government auditing standards (GAGAS) by DoD audit organizations. Our narrative response to your recommendations are included at Enclosure 1. As an alternative to the recommendation that this office rescind or modify our May 2007 External Review of the DCAA Quality Control System opinion report, we instead notified DCAA on August 24, 2009, that our May 2007 "adequate" opinion on the DCAA system of quality control would expire on August 26, 2009 (See Enclosure 2). On the basis of our action, DCAA has begun to qualify its GAGAS-compliant audits with a statement noting an exception to compliance with the Quality Control and Assurance Standard. In addition, it was recommended that DCAA publicly disclose the concerns of your office to include questioning the reliability of audit reports issued during the period covered by our May 2007 opinion. Our August 24, 2009, memorandum to the DCAA will he made public upon the release of your report. Please contact Ms. Carolyn R. Davis at (703) 604-8877 if you have any questions. Sincerely, Signed by: Gordon S. Heddell: Enclosures: As stated: [End of letter] Enclosure 1: GAO Draft Report Dated July 31, 2009 GAO-09-468 (GAO Code 195099) "DCAA Audits: Widespread Problems With Audit Quality Require Significant Reform" Department Of Defense Inspector General Comments To The GAO Recommendations: Recommendation 1: The GAO recommends that the Department of Defense Inspector General (DoD IG) reconsider its overall conclusions in the May 2007 DoD IG report on the audit of DCAA's quality control system in which it reported an adequate ("clean") opinion on DCAA system of quality control in light of the serious deficiencies and findings included in that report and the additional evidence identified in our audit. (Page 57/GAO Draft Report) DODIG Response: Concur. Recommendation 2: The GAO recommends that the Department of Defense Inspector General based on the above, determine whether the report should be rescinded or modified. (Page 57/GAO Draft Report) DODIG Response: Nonconcur. In reference to your recommendation that we rescind or modify our May 2007 External Review of the DCAA Quality Control System opinion report, we have taken an alternative action that conforms to the intent of your recommendations. The DoD IG has taken the extraordinary action of notifying DCAA that our "adequate" opinion on DCAA's system of quality control as detailed in the May 2007 External Review of the Defense Contract Audit Agency (DCAA) Quality Control System report will expire as of August 26, 2009 (See Enclosure 2). Our action recognizes that our May 2007 report was based upon a quality review methodology that differed substantially from your audit, but acknowledges that additional concerns about DCAA quality controls must be addressed. On August 5, 2009, we announced a project under the title "Evaluation of the Defense Contract Audit Agency Quality Assurance Program" (Project No. D2009-DIPOAC-0283). The results of this evaluation will be used in formulating our next opinion on the DCAA external quality control system for the period ending September 30, 2009. The overall opinion will take into consideration repeated non- compliances with Government Auditing Standards identified in our May 2007 and December 2003 opinion reports on the DCAA quality control system. By not rescinding our May 2007 report, we can also use the repeat deficiencies in formulating our next opinion. Placing an expiration date on the opinion means that after August 26, 2009, DCAA will be operating without an opinion on its system of quality control. We have recommended to the Director, DCAA, that her agency immediately begin to qualify its GAGAS-compliant audits with a modified GAGAS statement noting an exception to compliance with the Quality Control and Assurance Standard. Additionally, we have recommended that DCAA publicly disclose on its official website your concerns, including the questioning of the reliability of audit reports issued during the period covered by this office's May 2007 opinion. Our May 2007 report and the GAO July 2009 draft report focused on the period ended September 30, 2006. 1 believe it is paramount that we move forward and critically analyze the existing DCAA environment, its organizational effectiveness, and the quality of its recent audit work. Given the critical mission of DCAA, we plan to focus our attention and resources on a current analysis to yield real-time effective recommendations to improve DCAA audit quality. [End of Enclosure 1] Enclosure 2: Inspector General: Department Of Defense: 400 Army Navy Drive: Arlington, Virginia 22202-4704: August 24, 2009: Memorandum For Director, Defense Contract Audit Agency: Subject: Review of the Defense Contract Audit Agency Quality Control System (OIG Report No. D-2007-6-006) and GAO Draft Report, "DCAA Audits: Widespread Problems with Audit Quality Require Significant Reform" (GAO-09-468): Our May 2007 External Review of the Defense Contract Audit Agency (DCAA) Quality Control System provided DCAA with an adequate opinion on its internal quality control system while still identifying significant audit quality problems. However, our significant findings in 2007 coupled with the results of the July 2009 Government Accountability Office (GAO) draft report, "DCAA Audits: Widespread Problems with Audit Quality Require Significant Reform" (GAO-09-468), necessitates that we take further action. We have determined that it is not prudent to allow the adequate opinion from our May 2007 report to carry forward. Therefore, effective August 26, 2009, our adequate opinion will no longer apply to the DCAA system of quality control and will require further actions on your part. Specifically, we recommend the following be implemented immediately: 1. After August 26, 2009, all DCAA audits identified as being in compliance with generally accepted government auditing standards (GAGAS) must be qualified with a modified GAGAS statement noting an exception to compliance with the Quality Control and Assurance standard. 2. DCAA should publicly disclose on the DCAA website GAO's concerns regarding the reliability of DCAA audit reports issued during the period covered by this office's May 2007 opinion. As requested by your July 30, 2009, letter to my office, we have started preparing for the external quality control review ("peer review") of DCAA for the period ending September 30, 2009. On August 5, 2009, we announced that project under the title "Evaluation of the Defense Contract Audit Agency Quality Assurance Program" (Project No. D2009-DIPOAC-0283.000). The results of this evaluation will be used in formulating our overall opinion of the DCAA quality control system. The overall opinion will also take into consideration repeated non- compliances with Government Auditing Standards identified in our May 2007 and December 2003 opinion reports on the DCAA quality control system. This action is necessary to maintain the integrity of this office's oversight responsibilities for audits conducted by Department of Defense agencies. This memorandum is being distributed to the recipients of our May 1, 2007, opinion report, and will be posted with the May 2007 opinion report on our DoDIG website. Please contact me or Mr. Charles W. Beardall at (703) 602-1017 or Ms. Carolyn R. Davis at (703) 604-8877 if you have any questions. Signed by: Gordon S. Heddell: cc: Under Secretary of Defense (Comptroller)/Chief Financial Officer: [End of Enclosure 2] [End of section] Footnotes: [1] GAO, DCAA Audits: Allegations That Certain Audits at Three Locations Did Not Meet Professional Standards Were Substantiated, [hyperlink, http://www.gao.gov/products/GAO-08-993T] (Washington, D.C.: Sept. 10, 2008). [2] GAO, DCAA Audits: Allegations That Certain Audits at Three Locations Did Not Meet Professional Standards Were Substantiated, [hyperlink, http://www.gao.gov/products/GAO-08-857] (Washington, D.C.: July 22, 2008). [3] Hereafter referred to as the DOD Comptroller/CFO. [4] Under Secretary of Defense--Comptroller, Memorandum for Director Defense Contract Audit Agency, Subject: Implementation of Corrective Actions, (Washington, D.C.: Aug. 20, 2008). [5] Defense Business Board, Report to the Secretary of Defense: Independent Review Panel Report on the Defense Contract Audit Agency, October 2008. [6] GAO, Generally Accepted Government Auditing Standards, [hyperlink, http://www.gao.gov/products/GAO-03-673G] (Washington, D.C.: June 2003) and GAO-07-731G (Washington, D.C.: July 2007). [7] In the case of follow-up audits, we also reviewed the documentation for the previous audit to gain an understanding of the scope of work and deficiencies identified in the prior audit. [8] In selecting the seven DCAA offices, we considered a 2-year history of internal control audit results. The seven DCAA offices we selected reported adequate opinions on 89 percent or more of the internal control reports they issued during fiscal year 2006. During fiscal year 2005, 4 of the 7 offices reported adequate opinions in 85 percent or more of the internal control reports they issued, and the other 3 offices issued adequate opinions in 50 to 69 percent of the internal control audit reports they issued. [9] DCAA, Contract Audit Manual (CAM) 5-1202.1a and Defense Federal Acquisition Regulation Supplement (DFARS) 215.407-5. [10] FAR 16.104(h) and 16.301-3(a)(1). [11] FAR 42.101 and DFARS 242.803. [12] Contractor overpayments can occur as a result of errors made by paying offices, such as duplicate payments and payments in excess of amounts billed, and contractor billing errors, such as using the wrong overhead rate, failing to withhold designated amounts on progress payments, duplicate billings, or billing for unallowable cost. Recoveries of overpayments can be accomplished through refunds, subsequent billing offsets, or other adjustments to correct billing errors. [13] Although we selected 73 assignments for review, two internal control assignments were assist audits and two cost related assignments were not completed assignments. As a result, we did not consider these four assignments in our analysis, and we discuss the results of our analysis of the 69 completed assignments that we reviewed. [14] On August 19, 2008, at the request of the DOD Defense, Comptroller, the Deputy Secretary of Defense established an independent review panel under the Defense Business Board (DBB) to review DCAA operations and make recommendations for improvements. [15] Codified in an appendix to Title 5 of the United States Code (hereafter 5 U.S.C. App.). [16] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [17] GAO, High-Risk Series: An Update, GAO-09-271 (Washington, D.C.: January 2009). [18] GAO, Global War on Terrorism: DOD Needs to More Accurately Capture and Report the Costs of Operation Iraqi Freedom and Operation Enduring Freedom, [hyperlink, http://www.gao.gov/products/GAO-09-302] (Washington, D.C.: Mar. 17, 2009); Centers for Medicare and Medicaid Services: Internal Control Deficiencies Resulted in Millions of Dollars of Questionable Contract Payments, [hyperlink, http://www.gao.gov/products/GAO-08-54] (Washington, D.C.: Nov. 15, 2007); Defense Contract Management: DOD's Lack of Adherence to Key Contracting Principles on Iraq Oil Contract Put Government Interests at Risk, [hyperlink, http://www.gao.gov/products/GAO-07-839] (Washington, D.C.: July 31, 2007); Hanford Waste Treatment Plant: Department of Energy Needs to Strengthen Controls over Contractor Payments and Project Assets, [hyperlink, http://www.gao.gov/products/GAO-07-888] (Washington, D.C.: July 20,2007); Iraq Contract Costs: DOD Consideration of Defense Contract Audit Agency's Findings, [hyperlink, http://www.gao.gov/products/GAO-06-1132] (Washington, D.C. Sept. 25, 2006); Department of Energy, Office of Worker Advocacy: Deficient Controls Led to Millions of Dollars in Improper and Questionable Payments to Contractors, [hyperlink, http://www.gao.gov/products/GAO-06-547] (Washington, D.C.: May 31, 2006); and Federal Bureau of Investigation: Weak Controls over Trilogy Project Led to Payment of Questionable Contractor Costs and Missing Assets, [hyperlink, http://www.gao.gov/products/GAO-06-306] (Washington, D.C.: Feb. 28, 2006). [19] GAO, Defense Acquisitions: Assessments of Selected Weapon Programs, [hyperlink, http://www.gao.gov/products/GAO-09-326SP] (Washington, D.C.: Mar. 30, 2009); Defense Management: Actions Needed to Overcome Long-standing Challenges with Weapon Systems Acquisition and Service Contract Management, [hyperlink, http://www.gao.gov/products/GAO-09-362T] (Washington, D.C.: Feb. 11, 2009); and Defense Acquisitions: DOD's Increased Reliance on Service Contractors Exacerbates Long-standing Challenges, [hyperlink, http://www.gao.gov/products/GAO-08-621T] (Washington, D.C.: Jan. 23, 2008). [20] DCAA, Contract Audit Manual (CAM), DCAAM 7640.1. [21] Project 60 also resulted in consolidation of the military services' contract management activities under the Defense Contract Management Agency (DCMA), formerly the Defense Contract Management Command (DCMC) within the Defense Logistics Agency. On March 27, 2000, DCMC was established as DCMA under the authority of the Under Secretary of Defense (Acquisition, Technology, and Logistics). [22] DOD, General Plan: Consolidation of Department of Defense Contract Audit Activities into the Defense Contract Audit Agency (Feb. 17, 1965). [23] DODD 5105.36, paragraph 4.2, reissued on February 28, 2002. [24] DODD 5105.36, paragraphs 5.1 through 5.14. [25] Contract administration responsibilities are set forth in FAR Subparts 42.2 and 42.3. [26] CAM 2-001. [27] Disbursing officers are authorized to make payments on the authority of a voucher certified by an authorized certifying officer, who is responsible for the legality, accuracy, and propriety of the payment. 31 U.S.C. 3325, 3521(a), and 3528(a). [28] FAR 32.503-4. [29] CAM, 2-101. Except where stated otherwise in this report, various types of evaluations entailing different levels of assurance that DCAA refers to as audits--such as examinations, attestations, and reviews-- were subject to GAGAS. [30] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 1.01, and [hyperlink, http://www.gao.gov/products/GAO-07-731G], 1.03. [31] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 1.11. [32] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.01, and [hyperlink, http://www.gao.gov/products/GAO-07-731G], 6.01. [33] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 2.06 through 2.10. [34] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 2.01. [35] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 2.10. [36] According to documentation provided by DCAA as of the end of July 2009, the 80 rescinded reports include 62 reports related to findings in our July 2008 investigative report and 18 reports related to this audit. [37] See [hyperlink, http://www.gao.gov/products/GAO-03-673G], 3.19, and [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.10. [38] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.04b. [39] AICPA Statements on Auditing Standards, AU 350, and Audit and Accounting Guide: Audit Sampling, 3.14, 3.29-3.34, 3.58, and 3.61. [40] GAO/PCIE, Financial Audit Manual, GAO-08-585G (Washington, D.C.: July 2008). [41] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 4.15. [42] FAR 16.104(h) and 16.301-3(a)(1). [43] FAR 42.101 and DFARS 242.803(b)(i)(C). [44] DFARS 215.407-5-70; see FAR 15.407-5. [45] DCAA, "Audit Guidance on Significant Deficiencies/Material Weaknesses and Audit Opinions on Internal Control Systems," 08-PAS-043R (Dec. 19, 2008). [46] Under its decentralized management environment, DCAA headquarters obtains field office agreement to rescind audit reports that do not meet GAGAS. [47] DOD, Fiscal Year 2008 Agency Financial Report, Department of Defense (Washington, D.C.: Nov. 17, 2008). [48] DCAA does not perform paid voucher reviews during the year that it performs an audit of the contractor's billing system internal controls. [49] CAM 6-1007. [50] AU 350.19 and SSAE 15.64 and 15.69. [51] AU 350.07 through 350.14. [52] [hyperlink, http://www.gao.gov/products/GAO-08-585G], 450. [53] Confidence interval is the probability associated with the precision, that is, the probability that the true misstatement is within the confidence interval. [54] For example, for a confidence level of 90 percent and a tolerable rate of 5 percent, a sample size of 45 transactions would have an acceptable number of deviations of zero and a sample size of 78 transactions would have an acceptable number of deviations of one. For the same confidence level of 90 percent and a tolerable rate of 10 percent, a sample size of 45 would have an acceptable number of deviations of one and a sample size of 78 would have an acceptable number of deviations of four. [55] The AICPA Audit Guide is an interpretive publication pursuant to AT section 50, SSAE Hierarchy (AICPA, Professional Standards, vol. 1). [56] AICPA Audit Guide I-17, and AU 350.23. Statements on Auditing Standards (SAS) 39 is referred to as AU 350. [57] DCAA, "Audit Program: Audit of Contractor Overpayments," (Activity Code 17310), April 2004, September 2007, and May 2008. [58] CAM 6-102. [59] FAR 42.101, DFARS 242.803(b)(1)(c), and CAM 6-1007. [60] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 3.03 through 3.18. [61] Canceling funds refers to the point in time at which the availability of a fixed-year appropriation cancels and is no longer available for recording, adjusting, and liquidating obligations properly chargeable to the appropriation. (31 U.S.C. 1552(a) and 1553(b)). [62] Pub. L. No. 103-62, 107 Stat. 285 (Aug. 3, 1993). [63] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.50- 3.52. [64] In using professional judgment, GAGAS [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.32 and 3.35, require auditors to act diligently in accordance with applicable professional standards and ethical principles in all aspects of carrying out their professional responsibilities. [65] All 10 categories of recommendations in the DOD IG's report related to GAGAS compliance problems. [66] DOD Inspector General, Oversight Review: Review of the Defense Contract Audit Agency Quality Control System, Report No. D-2007-6-006 (Arlington, VA: May 1, 2007). [67] Of the 80 rescinded audit reports, 39 reports were issued in fiscal year 2006--the period covered in the DOD IG peer review report on DCAA. [68] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [69] Codified, as amended, at 31 U.S.C. ch. 75. The Single Audit Act requires that a state, local government or non-profit organization that expends more than $500,000 in a fiscal year undergo a single audit, which includes an audit of the entity's financial statements and Schedule of Expenditures of Federal Awards, as well as testing of and reporting on certain internal controls. [70] [hyperlink, http://www.gao.gov/products/GAO-07-731G] 3.40 through 3.49, and [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [71] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.43. [72] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 60.04a, and [hyperlink, http://www.gao.gov/products/GAO-07-731G], 6.04a. [73] [hyperlink, http://www.gao.gov/products/GAO-08-857]. [74] [hyperlink, http://www.gao.gov/products/GAO-08-857]. [75] We spoke to the auditors and reviewed documentation they provided. To the extent that the auditors also submitted complaints to DCAA's anonymous Web site, we reviewed DCAA's handling of their complaints. [76] After we provided our report to DOD for comment, we received updated information on DCAA anonymous Web site complaints. As of the end of July 2009, DCAA had established 209 cases. Eight of those cases were immediately referred to the DOD IG for investigation. Of the 209 cases, 82 were for the Western Region. [77] [hyperlink, http://www.gao.gov/products/GAO-08-857]. [78] Based on audit quality problems identified in our July 2008 report, in August 2008, the DOD Comptroller/CFO conducted a tiger team review in August 2008 and also asked the Secretary of Defense for support in conducting a study of DCAA. With the Secretary's approval, the DOD Advisory Panel determined that the Defense Business Board would perform this study. [79] On most contracting matters with DCAA involvement, the cognizant agency contracting officer makes final decisions based on DCAA's findings and recommendations. [80] DCAA also established contracting officer sustention rates related to questioned cost and net savings as an informational goal to show return to the taxpayer. [81] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.40. [82] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.49- 3.52. [83] 10 U.S.C. 2313 and 2306a. [84] FAR 52.214-26, 52.215-2. [85] The Army Force Management Support Agency's mission includes providing requirements studies and staffing analysis as well as determining whether organizations have the appropriate staff to carry out their mission. The Army Force Management Support Agency also provides services to DOD components. [86] Pub. L. No. 110-181, 1705, 122 Stat. 3 (Jan. 28, 2008), the National Defense Authorization Act for Fiscal Year 2008, authorized the Secretary to establish the Department of Defense Acquisition Workforce Fund, in addition to other funds that may be available, for the recruitment, training, and retention of department acquisition personnel. The fund is managed by the Under Secretary of Defense for Acquisition, Technology, and Logistics. [87] According to a DOD Comptroller official, DCAA will receive an additional 300 positions in fiscal year 2009 and additional 200 positions in each of fiscal years 2010 and 2011. [88] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.38 and AU 339.12. [89] Disbursing officers are authorized to make payments on the authority of a voucher certified by an authorized certifying officer, who is responsible for the legality, accuracy, and propriety of the payment. 31 U.S.C. 3325, 3527(c). DOD 7000.14-R, Department of Defense Financial Management Regulation (DFMR), Vol. 5, Ch. 11 (March 2009), paras. 110102, 110203. In general, certifying officers designated in writing by the agency are financially liable for any improper, illegal, or incorrect payment made, and each payment made must be audited (or "examined"). 31 U.S.C. 3521(a), 3528(a). DFMR, Vol. 5, Ch. 33 (April 2005), para. 330303. However, 31 U.S.C. 3521(b) authorizes heads of agencies to carry out a statistical sampling procedure, within certain parameters, to audit vouchers when the head of the agency determines that economies will result. Further, 31 U.S.C. 3521(c) provides that certifying and disbursing officials are not liable for payments that are not audited if they were made in good faith under a statistical sampling procedure. See 68 Comp. Gen. 618 (1989); also see generally, GAO, Policy and Procedures Manual for Guidance of Federal Agencies, title 7, 6.5, 7.4, and 7.5 (Washington, D.C.: May 18, 1993). [90] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [91] The Baldrige National Quality Program is named for Malcolm Baldrige, a former Secretary of Commerce, who was a proponent of quality management as a key to national prosperity and long-term strength. The seven Baldrige performance excellence criteria are: leadership; strategic planning; customer and market focus; measurement, analysis, and knowledge management; workforce focus; process management; and results. [92] Codified in an appendix to Title 5 of the United States Code (hereafter 5 U.S.C. App.). [93] The IG Act also requires the heads of many "designated federal entities" to appoint an inspector general for each entity. 5 U.S.C. App. 8G. [94] 5 U.S.C. App. 3(a). [95] 5 U.S.C. App. 3(b). [96] 5 U.S.C. App. 8F(4)(A). [97] 5 U.S.C. App. 6(f)(1). [98] 5 U.S.C. App. 6(a)(1), (4), and (5). [99] As noted previously, in these cases, there was no evidence that DCAA supervisors elevated the issue to management or to procurement officials to initiate enforcement action, as set out in DCAA policy. [100] 5 U.S.C. App. 5(a). [101] GAO, Defense Contract Audits: Current Organizational Relationships and Responsibilities, [hyperlink, http://www.gao.gov/products/GAO/AFMD-91-14] (Washington, D.C.: Apr. 3, 1991). [102] The Deputy Secretary of Defense is appointed by the President after confirmation by the Senate. 10 U.S.C. 132(a). Among other duties as assigned by the Secretary of Defense and in statute, the Deputy Secretary serves as the Chief Management Officer of the department with primary responsibility for "effectively and efficiently organizing the business operations of the Department of Defense." Pub. L. No. 110-181, div A, title IX, 904(a)(2) (Jan. 28, 2008). In that capacity, the Deputy Secretary is to be assisted by a Deputy Chief Management Officer, who also is appointed by the President after confirmation by the Senate, and who supervises the Defense Business Transformation Agency. 10 U.S.C. 132(c), 192(e). [103] Current provisions of law relevant to the Secretary of Defense establishing and assigning defense agencies and defense field activities within the Office of the Secretary of Defense include 10 U.S.C. 125(a), 131(b), 191(b), 192(a), and 194. [104] Pub. Law No. 111-23, 123 Stat. 1704, May 22, 2009. [105] The USD(AT&L) is responsible under 10 U.S.C. 133 for establishing DOD policies related to the negotiation, award, and administration of contracts, such as those related to the use of contract audit services, and for coordinating contract audit activities within DOD. [106] [hyperlink, http://www.gao.gov/products/GAO-08-857]. [107] See [hyperlink, http://www.gao.gov/products/GAO-07-731G], 1.03. [108] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 2.06. [109] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.55. [110] CAM 2-101(a) and 2-103c(1). [111] We originally selected 39 internal control audits for our review. Because two audit assignments were performed as assist audits to an internal control audit in our selection, we considered these three assignments as one audit, and therefore, we reviewed a total of 37 audits of contractor system internal control audits. [112] [hyperlink, http://www.gao.gov/products/GAO-03-673G], Chapter 3, especially 3.03, 3.04, and 3.13-3.17. [113] GAGAS for attestations audits related to requirements 2 through 8 are covered in [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.04a; 6.13-6.14; 6.24a & c; 6.15a, and 6.16-6.20; 6.02a, 6.04b, 6.22 and 6.24; and 6.28-6.54. [114] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 3.33- 3.38. [115] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 3.03, and [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.02. [116] See [hyperlink, http://www.gao.gov/products/GAO-03-673G], 3.19 and [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.10. [117] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 3.16. [118] See [hyperlink, http://www.gao.gov/products/GAO-03-673G], 3.19, and [hyperlink, http://www.gao.gov/products/GAO-07-731G], 3.10. [119] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.04a, 6.06, and 6.11. [120] CAM 5-103. [121] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.13- 6.14. [122] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.13 and 6.14. [123] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.15a. [124] CAM, Figure 4-7-3. [125] DOD Inspector General, Handbook on Fraud Indicators for Contract Auditors, Section II.4 (IGDH 7600.3 APO, March 31, 1993). [126] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.04b, 6.22, and 6.24a. [127] CAM 4-600 and Appendix B. [128] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.04b. [129] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.24. [130] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.04a and 6.24e. [131] AICPA, Standards for Attestation Engagements, AT 101.63 incorporated by reference in [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.01, and [hyperlink, http://www.gao.gov/products/GAO-07-731G], 6.01. [132] [hyperlink, http://www.gao.gov/products/GAO-07-731G], 7.37. [133] [hyperlink, http://www.gao.gov/products/GAO-03-673G], 6.50 and 6.24, and [hyperlink, http://www.gao.gov/products/GAO-07-731G], 6.24. [134] We initially selected 34 cost-related audit assignments for review. After reviewing the audit documentation, we determined that one assignment only covered part of an audit and the other assignment was terminated and the procedures were incorporated into a related billing system audit. Therefore, we reviewed a total of 32 completed cost- related assignments. [135] Contractor overpayments can occur as a result of errors made by paying offices, such as duplicate payments and payments in excess of amounts billed, and contractor billing errors, such as using the wrong overhead rate, failing to withhold designated amounts on progress payments, duplicate billings, or billings for unallowable cost. Recoveries of overpayments can be accomplished through refunds, subsequent billing offsets, or other adjustments to correct billing errors. Unallowable costs include lobbying cost, certain legal expenses, executive and management bonuses, luxury items, and certain overhead costs. [136] REA relates to contractor requests to adjust contract terms for rates and payments resulting from contract modifications. In the case of the two REA audits, contract modifications related to requests for increased hours of service and related labor and materials. [137] FAR 42.101 and DFARS 242.803. [138] FAR 52.232-25(d) was amended in October 2008 to require contractors to monitor for and make adjustments to correct overpayments they may receive, but it still does not specify a timeframe for making any needed adjustments. [139] FAR 32.606(a). [140] DCAA, Audit of Contract Overpayments Audit Program, version 2.1, October 2006. [141] Although the government pays contractor invoices on a provisional basis when they are submitted for payment, DCAA incurred cost audits provide the basis for final approval of contractor incurred costs claims. [142] Questioned costs include costs questioned by DCAA auditors as unallowable or unsupported. [143] [hyperlink, http://www.gao.gov/products/GAO-08-857]. [144] As stated in DCAA's Contract Audit Manual, CAM 2-100, DCAA uses the term audit to refer to a variety of audits, evaluations, reviews, assessments, and analyses. [145] GAO, Government Auditing Standards: 2003 Revision, [hyperlink, http://www.gao.gov/products/GAO-03-673G] (Washington, D.C: June 2003) and Government Auditing Standards: 2007 Revision, [hyperlink, http://www.gao.gov/products/GAO-07-731G] (Washington, D.C: July 2007). [146] In selecting the seven DCAA offices, we considered a 2-year history of internal control audit results. The seven DCAA offices we selected reported adequate opinions on 89 percent or more of the internal control reports they issued during fiscal year 2006. During fiscal year 2005, four of the seven offices reported adequate opinions in 85 percent or more of the internal control reports they issued, and the other 3 offices issued adequate opinions in 50 to 69 percent of the internal control audit reports they issued. [147] DCAA Contract Audit Manual (CAM) 5-1202.1a and Defense Federal Acquisition Regulation Supplement (DFARS) 215.407-5. [148] FAR 16.301-3(a)(1). [149] FAR 42.101, and DFARS 242.803. [150] In the case of follow-up audits, we also reviewed the documentation for the previous audit to gain an understanding of the scope of work and deficiencies previously identified. [151] The Internal Control Integrated Framework developed by the Committee on Sponsoring Organizations (COSO) of the Treadway Commission, September 1993, are applicable to private sector entities. We considered whether DCAA audits addressed contractor controls related to the five key control activities: (1) contractor control environment; (2) contractor risk assessment; (3) control activities, including policies and procedures and segregation of duties; (4) information and communication (i.e., information system processing controls); and (5) monitoring. [152] Contractor overpayments can occur as a result of errors made by paying offices, such as duplicate payments and payments in excess of amounts billed, and contractor billing errors, such as using the wrong overhead rate, failing to withhold designated amounts on progress payments, duplicate billings, or billing for unallowable cost. Recoveries of overpayments can be accomplished through refunds, subsequent billing offsets, or other adjustments to correct billing errors. Unallowable cost include lobbying cost, certain legal expenses, executive and management bonuses, luxury items, and certain overhead costs. [153] REA audits relate to reviewing contractor requests for adjustments in billing rates pursuant to contract modifications. For example, if a contractor is asked to provide additional services or expand hours of service, contract costs would need to be recalculated and adjusted rates verified. REA audits relate to audits of contractor estimating system controls. [154] Pub. L. No. 103-62, 107 Stat. 285 (Aug. 3, 1993). [155] [hyperlink, http://www.gao.gov/products/GAO-03-673G], and [hyperlink, http://www.gao.gov/products/GAO-07-731G]. [156] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-21.32.1] (Washington, D.C.: November 1999). [157] DOD Inspector General, Oversight Review: Review of the Defense Contract Audit Agency Quality Control System, Report No. D-2007-6-006 (Arlington, VA: May 1, 2007). [158] [hyperlink, http://www.gao.gov/products/GAO-03-673G], and [hyperlink, http://www.gao.gov/products/GAO-07-731G]. [159] 5 U.S.C., App. [160] GAO, Defense Contract Audits: Current Organizational Relationships and Responsibilities, [hyperlink, http://www.gao.gov/products/GAO/AFMD-91-14] (Washington, D.C.: Apr. 3, 1991). [161] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO‘s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO‘s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.