Homeland Security

Successes and Challenges in DHS's Efforts to Create an Effective Acquisition Organization Gao ID: GAO-05-179 March 29, 2005

Department of Homeland Security (DHS) organizations are expected to work together to protect the United States from terrorism. To support this primary mission, DHS has been acquiring billions of dollars worth of goods and services. DHS also has been working to integrate the disparate acquisition processes and systems that organizations brought with them when DHS was created 2 years ago. GAO was asked to identify (1) areas where DHS has been successful in promoting collaboration among its various organizations and (2) areas where DHS still faces challenges in integrating the acquisition function across the department. GAO was also asked to assess DHS's progress in implementing an effective review process for major, complex investments.

DHS's disparate organizations have quickly established collaborative relationships to leverage spending for various goods and services without losing focus on small businesses. DHS is using strategic sourcing, that is, formulating purchasing strategies to meet departmentwide requirements for specific commodities, such as office supplies, boats, energy, and weapons. By fostering collaboration, DHS has leveraged its buying power and savings are expected to grow. Also off to a good start is the small business program, whose reach is felt across DHS. Representatives have been designated in each DHS procurement office to help ensure that small businesses have opportunities to compete for DHS's contract dollars. In contrast, lack of clear accountability is hampering DHS's efforts to integrate the acquisition functions of its numerous organizations into an effective whole. DHS remains a collection of disparate organizations, many of which are performing functions with insufficient oversight, giving rise to an environment rife with challenges. Some of DHS's organizations have major, complex acquisition programs that are subject to a multitiered investment review process to help reduce risk and increase chances for successful outcomes in terms of cost, schedule, and performance. Part of the review process features a knowledge-based acquisition approach pioneered by successful commercial firms. DHS's adaptation of this best practices approach, however, does not require two critical management reviews and is missing some key information before decisions are made to invest additional resources. In addition, contractor tracking and oversight is not fully incorporated into DHS policy and guidance. Finally, some aspects of the review process--which has been under revision for many months--need clarification.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-05-179, Homeland Security: Successes and Challenges in DHS's Efforts to Create an Effective Acquisition Organization This is the accessible text file for GAO report number GAO-05-179 entitled "Homeland Security: Successes and Challenges in DHS's Efforts to Create an Effective Acquisition Organization' which was released on April 28, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Requesters: United States Government Accountability Office: GAO: March 2005: Homeland Security: Successes and Challenges in DHS's Efforts to Create an Effective Acquisition Organization: GAO-05-179: GAO Highlights: Highlights of GAO-05-179, a report to congressional requesters. Why GAO Did This Study: Department of Homeland Security (DHS) organizations are expected to work together to protect the United States from terrorism. To support this primary mission, DHS has been acquiring billions of dollars worth of goods and services. DHS also has been working to integrate the disparate acquisition processes and systems that organizations brought with them when DHS was created 2 years ago. GAO was asked to identify (1) areas where DHS has been successful in promoting collaboration among its various organizations and (2) areas where DHS still faces challenges in integrating the acquisition function across the department. GAO was also asked to assess DHS‘s progress in implementing an effective review process for major, complex investments. What GAO Found: DHS‘s disparate organizations have quickly established collaborative relationships to leverage spending for various goods and services without losing focus on small businesses. DHS is using strategic sourcing, that is, formulating purchasing strategies to meet departmentwide requirements for specific commodities, such as office supplies, boats, energy, and weapons. By fostering collaboration, DHS has leveraged its buying power and savings are expected to grow. Also off to a good start is the small business program, whose reach is felt across DHS. Representatives have been designated in each DHS procurement office to help ensure that small businesses have opportunities to compete for DHS‘s contract dollars. In contrast, lack of clear accountability is hampering DHS‘s efforts to integrate the acquisition functions of its numerous organizations into an effective whole. DHS remains a collection of disparate organizations, many of which are performing functions with insufficient oversight, giving rise to an environment rife with challenges, as shown in the following table. Challenges Facing DHS‘s Efforts to Integrate Acquisition Functions: Problem areas: Overall integration; Challenges: New policy emphasizes need for unified, integrated acquisition organization but allows U.S. Coast Guard and U.S. Secret Service to remain exempt from integration efforts. Possible workload imbalances have not been addressed, nor has the lack of enforcement of program managers‘ training and certification. Problem areas: Dual accountability; Challenges: Some of the primary duties delegated to the Chief Procurement Officer have also been given to heads of DHS‘s organizations, resulting in confusion over who is ultimately accountable for acquisition decisions. Problem areas: Chief Procurement Officer‘s oversight staff; Challenges: Office has lacked sufficient staff to ensure compliance with DHS‘s acquisition regulations and policies. Problem areas: Office of Procurement Operations‘ use of interagency contracting; Challenges: Created almost 1 year after DHS was formed to support the organizations that lacked their own procurement support, Procurement Operations lacks sufficient staff and relies heavily on interagency contracting, but missing are management controls to properly oversee this activity, including fees paid to other agencies. Source: GAO analysis. [End of table] Some of DHS‘s organizations have major, complex acquisition programs that are subject to a multitiered investment review process to help reduce risk and increase chances for successful outcomes in terms of cost, schedule, and performance. Part of the review process features a knowledge-based acquisition approach pioneered by successful commercial firms. DHS‘s adaptation of this best practices approach, however, does not require two critical management reviews and is missing some key information before decisions are made to invest additional resources. In addition, contractor tracking and oversight is not fully incorporated into DHS policy and guidance. Finally, some aspects of the review process”which has been under revision for many months”need clarification. What GAO Recommends: GAO recommends that the Secretary of Homeland Security consider adding resources to DHS‘s strategic sourcing program, correct deficiencies in departmentwide oversight of acquisition policies and procedures, add resources for contracting, and shore up its review process for acquiring major, complex investments. In written comments on a draft of this report, DHS agreed with the recommendations. www.gao.gov/cgi-bin/getrpt?GAO-05-179. To view the full product, including the scope and methodology, click on the link above. For more information, contact Michael J. Sullivan at (937) 258-7915 or sullivanm@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: Initiatives to Leverage Buying Power and Small Business Program Have Helped Foster Collaboration among DHS Organizations: Goal of Creating an Integrated Acquisition Organization Is Hampered by Unclear Policy Decisions and Staffing Disparities: Despite Adoption of Many Best Practices, Review Process for Major Investments Lacks Key Reviews and Some Management Controls: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: Comments from the Department of Homeland Security: Appendix III: Office of Procurement Operations' Customers: Appendix IV: Summary of Selected Commodity Council Strategic Sourcing Initiatives: Appendix V: Selected Acquisition Management Best Practices: Appendix VI: GAO Contacts and Staff Acknowledgments: Contacts: Staff Acknowledgments: Tables: Table 1: DHS's Principal Organizations and Their Missions: Table 2: Sampling Error at the 95 Percent Confidence Level for the Sample of Interagency Agreements: Table 3: Sampling Error at the 95 Percent Confidence Level for the Sample of Interagency Agreements Fee Payments: Figures: Figure 1: DHS's Directorates and Other Principal Organizations and Sources of Contracting Support: Figure 2: Reported Monthly Monetary Benefits Generated by DHS's Strategic Sourcing Program: Figure 3: Comparison of Fiscal Year 2004 Dollars Obligated per Contracting Staff within Each DHS Contracting Office: Figure 4: General Depiction of DHS's Investment Review Process for Major, Complex Investments: Figure 5: Applying the Knowledge-Based Approach to DHS's Acquisition Framework: Abbreviations: DHS: Department of Homeland Security: DOD: Department of Defense: MANPADS: man-portable air defense systems: US-VISIT: U.S. Visitor and Immigrant Status Indicator Technology: United States Government Accountability Office: Washington, DC 20548: March 29, 2005: The Honorable Tom Davis: Chairman: Committee on Government Reform: House of Representatives: The Honorable Susan M. Collins: Chairman: Committee on Homeland Security and Governmental Affairs: United States Senate: Since it was established, in March 2003,[Footnote 1] the Department of Homeland Security (DHS) has been faced with assembling 23 separate federal agencies and organizations with multiple missions, values, and cultures into one cabinet-level department.[Footnote 2] This mammoth task--one of the biggest mergers ever to take place within the federal government--involves a variety of transformational efforts, one of which is to design and implement the necessary management structure and processes for acquiring goods and services. In January 2003, we designated DHS's implementation and transformation as high-risk because of the size and complexity of the effort and the existing challenges faced by the components being merged into the department.[Footnote 3] As it progresses through the early stages of its merger and transformation, DHS has an opportunity to put into place the necessary elements to become a 21st century federal department with a high- quality acquisition organization that effectively supports its critically important missions. DHS has some of the most extensive acquisition needs within the U.S. government. In fiscal year 2004, the department obligated $9.8 billion to acquire a wide range of goods and services--such as information systems, new technologies, weapons, aircraft, ships, and professional services. At the same time, DHS is working to integrate the many acquisition processes and systems that the disparate agencies and organizations brought with them. Given DHS's complex merger, you asked us to review the department's progress in establishing an effective acquisition organization. In response to your request, we (1) identified areas where DHS has been successful in promoting collaboration among its various organizations and (2) identified areas where DHS still faces challenges in integrating the acquisition function across the department. We also assessed the department's progress in implementing an effective review process for its major, complex investments. To conduct this work, we assessed information from DHS headquarters and the department's principal organizations and compared the information against what our previous work has shown to be best acquisition practices. We reviewed agency directives, memorandums, and other documentation; interviewed agency officials; and analyzed agency systems and processes. For the purposes of this report, the term acquisition refers to the overall effort to acquire goods and services and involves a host of activities. Acquisition begins when an agency establishes its requirements and moves through a process that involves soliciting providers of goods and services, awarding contracts, monitoring performance, and handling various contract administration duties. In some cases, acquisition can include development of a new product, such as a weapon system or a database system. Acquisition also can be the simple purchase of existing products. The Federal Acquisition Regulation, which governs acquisitions within the federal government, defines the term procurement as being synonymous with acquisition.[Footnote 4] Appendix I presents our scope and methodology in more detail. We conducted our review from March 2004 through February 2005 in accordance with generally accepted government auditing standards. Results in Brief: In the relatively short time since its inception, DHS has demonstrated some successes in implementing a strategic sourcing program to leverage the department's buying power and in creating a small business program.[Footnote 5] Both of these efforts have fostered an environment in which DHS's various organizations work collaboratively toward a common goal. The cornerstone of the strategic sourcing program is the use of commodity councils, each consisting of a cross-functional group of acquisition-related personnel and specialists from the various organizations. These councils are charged with formulating purchasing strategies to meet departmentwide requirements for specific commodities, such as office supplies and the department's vehicle fleet. Even though the councils have faced challenges in gathering accurate spending data and in managing workload, they have helped spur collaboration and cooperation across the department. Moreover, DHS already has reported $14 million in dollar savings as a result of leveraging resources across DHS, and officials expect the savings to grow over time. Another area of early success is DHS's small business program, which has been proactive in working with each organization in the department to ensure that small businesses have the opportunity to compete for DHS dollars. In fiscal year 2004, DHS reported that 35 percent of its prime contract dollars went to small businesses, exceeding its goal of 23 percent. Notwithstanding these initial successes, DHS's progress in creating a unified acquisition organization has been slowed by policy decisions that create ambiguity and by procurement staffing disparities within the department. An October 2004 management directive emphasizes the need for a unified, integrated acquisition organization but relies on a system of dual accountability between the Chief Procurement Officer and the heads of the department's organizations to make this happen. The Chief Procurement Officer has been delegated the responsibility to manage, administer, and oversee all acquisition activity across DHS, but in practice enforcement of these activities is spread throughout the department with unclear accountability. Further, the directive states that the U.S. Coast Guard and U.S. Secret Service are statutorily exempt from its application. Although the Homeland Security Act provides that both the Coast Guard and the Secret Service shall be maintained as distinct entities within the department, we found no reasonable basis to conclude that the directive could not be made applicable to them. Rather, it appears to be a policy decision that is likely to hamper efforts to effectively integrate the acquisition function in DHS. To a great extent, the various acquisition organizations within the department are still operating in a disparate manner, with oversight of acquisition activities left primarily up to each individual organization. Significant disparities in the dollar value of contracting staff's workloads across DHS have only recently begun to be addressed. Staffing shortages have led one organization, which handles about $2 billion in obligations for various departmental organizations, to rely extensively on outside agencies for contracting support--often for a fee. We found that this office lacked adequate internal controls to provide oversight of this interagency contracting activity. To protect its major, complex investments, DHS's Office of the Chief Financial Officer has put in place a review process that adopts many best practices--that is, proven methods, processes, techniques, and activities--to help the department reduce risk and increase the chances for successful outcomes in terms of cost, schedule, and performance. One best practice is an overarching, knowledge-based acquisition approach pioneered by successful commercial companies. A general principle of this approach is that program managers should provide sufficient knowledge about important aspects of their programs at key points in the acquisition process, so that senior leaders are prepared to make a well-informed investment decision before an acquisition moves forward. DHS's framework includes key tenets of this approach but does not require two critical management reviews to ensure that resources match customer needs and that design performs as expected. Also missing is important information to help reduce risk and meet cost and delivery targets for major investments. In addition, DHS's review process does not fully address how program managers will conduct effective contractor tracking and oversight. We also found that program managers lacked sufficient guidance about how to navigate investment reviews. The review process has been under revision for many months, and DHS officials could not tell us when the process would be finalized. In this report, we are making recommendations to the Secretary of Homeland Security to help ensure that the department's strategic sourcing program maintains its current momentum, that acquisition integration efforts continue, and that DHS leadership has the information it needs to proactively manage risks that arise during the acquisition of major, complex systems. In written comments on a draft of this report, DHS concurred with the recommendations. DHS's comments are included in their entirety in appendix II. Background: The Homeland Security Act of 2002 created DHS, effective March 1, 2003, by merging agencies and organizations that specialize in one or more aspects of homeland security. Some of those specialties are intelligence analysis, law enforcement, border security, transportation security, biological research, critical infrastructure protection, and disaster recovery. The intent behind DHS's merger and transformation was to improve coordination, communication, and information sharing among the multiple federal agencies responsible for protecting the homeland. Critical to performing the homeland security mission is the effective interaction between and integration of these agencies and organizations. Table 1 shows DHS's eight principal organizations and their missions. Table 1: DHS's Principal Organizations and Their Missions: Principal organizations[A]: Border and Transportation Security Directorate; Missions: Ensures security of U.S. borders and transportation systems; Enforces the nation's immigration laws; Manages and coordinates port-of- entry activities and oversees protection of government buildings. Principal organizations[A]: Emergency Preparedness and Response Directorate; Missions: Prepares for catastrophes; Oversees federal government's national response and recovery strategy. Principal organizations[A]: Information Analysis and Infrastructure Protection Directorate; Missions: Identifies and assesses threats; Recommends measures necessary to protect key resources and critical infrastructure. Principal organizations[A]: Science and Technology Directorate; Missions: Coordinates DHS's efforts in research and development. Principal organizations[A]: Management Directorate; Missions: Administers DHS's budget, financial management systems, procurement activities, human resources functions, information technology systems, facilities management, and performance measurement efforts. Principal organizations[A]: U.S. Secret Service; Missions: Protects U.S. President and other designated personnel, as well as the country's currency and financial infrastructure, and provides security for designated national events. Principal organizations[A]: U.S. Coast Guard; Missions: Protects the public, the environment and U.S. economic interests in the nation's ports and waterways, coasts, international waters, or any maritime region as required to support national security; has terrorism, counternarcotics border protection roles; and prevents illegal incursion of U.S. exclusive economic zone. Principal organizations[A]: U.S. Citizenship and Immigration Services; Missions: Directs immigration benefit system and promotes citizenship values by providing immigration services, such as immigrant and nonimmigrant sponsorship; adjustment of status; work authorization and other permits; naturalization of qualified applicants; and asylum or refugee processing. Sources: DHS (data); GAO (analysis). [A] This table does not show the organizations that fall under each of the five directorates. This table also does not show all organizations that report directly to the DHS Secretary and Deputy Secretary, such as executive secretary, legislative affairs, public affairs, chief of staff, inspector general, and general counsel. [End of table] Of the 23 entities that joined DHS from other agencies, only 7 came with their own procurement support. Providing support to the other entities--as well as a number of newly created entities, such as the offices of the Chief Information Officer and Chief Financial Officer-- is an eighth office, the Office of Procurement Operations (Procurement Operations). That office was not created until January 2004, almost a year after DHS came into being. Appendix III lists all of the DHS organizations that receive contracting support from Procurement Operations. Figure 1 shows the sources of contracting support for DHS's principal organizations. Figure 1: DHS's Directorates and Other Principal Organizations and Sources of Contracting Support: [See PDF for image] [End of figure] To carry out acquisition effectively across a large federal organization requires an integrated structure with standardized policies and processes, the appropriate placement of the acquisition function within the department, leadership that fosters good acquisition practices, and a general framework that delineates the key phases along the path for a major acquisition. An effective acquisition organization has in place knowledgeable personnel who work together to meet cost, quality, and timeliness goals while adhering to guidelines and standards for federal acquisition. Initiatives to Leverage Buying Power and Small Business Program Have Helped Foster Collaboration among DHS Organizations: In the 2 years since its creation, DHS has realized some successes in opening the lines of communication among the various organizations within the department through its strategic sourcing and small business programs. Both of these efforts have involved every principal organization in DHS, along with strong involvement from the Chief Procurement Officer, and both have yielded positive results. DHS already has begun to demonstrate that its strategic sourcing program can foster collaboration across the department and at the same time maximize the department's overall buying power. DHS's small business program has a presence departmentwide, and according to DHS officials, the department exceeded its 23 percent small business goal for fiscal year 2004.[Footnote 6] Strategic Sourcing Program Has Encouraged Partnerships across DHS and Begun to Realize Savings: Under the authority of the Chief Procurement Officer, DHS created a strategic sourcing group in October 2003 to leverage departmentwide spending for various commodities. The group brought together diverse expertise from throughout DHS. To identify commodities with the most potential for savings, strategic sourcing officials conducted a spend analysis using available acquisition databases, such as the Federal Procurement Data System, and input from DHS senior management.[Footnote 7] The following 15 commodities were identified as having potential to leverage the department's buying power: * aviation, * boats, * business wireless communications, * copiers, * energy, * enterprise software agreements, * facilities, * facilities security, * vehicle fleets, * mail, * office supplies, * professional services, * training, * uniforms, and: * weapons. Consistent with best practices, the strategic sourcing group then established commodity councils composed of representatives from across DHS. The commodity councils were assigned responsibility for further collection and refinement of historical procurement data in order to better assess future purchasing strategies. Typically, members from the strategic sourcing group and the DHS organization with the most expertise in a particular commodity serve as council cochairs. For example, a Coast Guard official is a cochair for the boats commodity council. Commodity council cochairs said they were willing to devote time to the strategic sourcing initiatives because they recognized the unique opportunity DHS had to move forward to leverage buying power across the department. Further, the cochairs were virtually unanimous in telling us that the councils enable stakeholders to build awareness of a particular commodity and develop strong relationships throughout DHS. They said the councils foster a sense of community in which the various organizations can share information, participate in forums, find commonalities, engage in open and productive communication, and make smarter and more collaborative business decisions. For example, the weapons commodity council routinely shares information on ammunition. When one DHS organization is low on ammunition, others help meet the need. Appendix IV contains more detail on initiatives that several commodity councils have undertaken. In fiscal year 2004, 4 commodity councils--office supplies, boats, energy, and weapons--reported approximately $14.1 million in cost savings and cost avoidances, and department officials expect the savings to continue to grow.[Footnote 8] The savings have resulted from DHS negotiating lower rates with suppliers and leveraging resources across the department. Figure 2 depicts the savings trend over a 12- month period. The September 2004 surge resulted when authorized DHS employees began purchasing pistols through two large contracts, a strategy spearheaded by the weapons commodity council. Figure 2: Reported Monthly Monetary Benefits Generated by DHS's Strategic Sourcing Program: [See PDF for image] [End of figure] Baseline Data and Workload Demands Have Presented Challenges: Some councils are encountering a problem faced by many federal departments and agencies, namely, a shortage of comprehensive data upon which to draw an accurate and detailed picture of what is being spent on certain commodities over time. Strategic sourcing officials and commodity council members told us that they cannot take full advantage of spend analyses, nor can they accurately chart historical spending, because DHS's acquisition databases do not contain enough procurement data. The problem is compounded by the fact that when parts of existing agencies, such as the Immigration and Naturalization Service from the Department of Justice, joined DHS, detailed information on its spending history was not available. Without an accurate analysis of how DHS organizations historically purchased a commodity, council members will likely continue to rely on a patchwork of estimates, as well as information from suppliers, to glean information on spending history and develop purchasing strategies for the future. In addition, some commodity council members have found it challenging to balance council duties with the demands and responsibilities of their full-time positions within DHS. Officials told us that council meetings and activities sometimes stall because council members must shift attention to their full-time positions. Many commodity councils did not make much progress during the last month of fiscal year 2004, we were told, because council members' time was diverted to year-end priorities. Our prior work on strategic sourcing shows that leading commercial companies often establish full-time commodity managers to more effectively manage commodities.[Footnote 9] Commodity managers help commodity councils define requirements with internal clients, negotiate with potential vendors, and resolve performance or other issues arising after a contract is awarded and can help maintain consistency, stability, and a long-term strategic focus. Small Business Program Off to Good Start: DHS's small business program has also had initial successes, reporting that 35 percent of fiscal year 2004 obligations were awarded to small business prime contractors, exceeding the department's goal of 23 percent. Although reporting directly to the Deputy Secretary of DHS, the Director of the Office of Small and Disadvantaged Business Utilization works closely with the Chief Procurement Officer to emphasize throughout the department the important public policy objective of small business inclusion in acquisition activities. The small business office, in conjunction with the procurement staff across the department, has created an outreach program that advises small businesses on ways to market goods and services to DHS. Small business representatives have been designated in each DHS procurement office, and each office is required to submit a forecast of upcoming contract opportunities above $100,000. DHS posts this information on a Web site so that small businesses can identify opportunities to do business with the department. The small business office has conducted extensive outreach to DHS's business partners through regular seminars and has established a mentor-protégé program that is designed to motivate and encourage large businesses to provide mutually beneficial developmental assistance to small businesses. The Director of the Office of Small and Disadvantaged Business Utilization and his staff have also been directly involved in DHS's strategic sourcing efforts to help ensure that, even as the department leverages its buying power, small businesses continue to have opportunities to compete for contracts. Several commodity councils have developed strategies to address this issue. The office supplies council has worked out an arrangement for DHS employees to purchase office supplies from the Department of Defense's (DOD) Web-based Emall, where employees can easily identify and order from small businesses.[Footnote 10] The head of procurement in the Immigration and Customs Enforcement organization, working closely with the weapons commodity council, awarded a contract for half of the largest pistol procurement in the history of U.S. law enforcement to a small business. According to the chair of the boats council, the council plans to consult with the Coast Guard's small business specialist to explore future possibilities for providing opportunities to small businesses. Goal of Creating an Integrated Acquisition Organization Is Hampered by Unclear Policy Decisions and Staffing Disparities: DHS's goal of integrating the acquisition function more broadly across the department has not been accomplished, and the introduction of a new policy has been unsuccessful in breaking down barriers to effective departmentwide management. An integrated acquisition organization is essential to the department's success in executing policies and processes to effectively obligate and administer billions of dollars in acquiring what DHS needs to accomplish its mission. An October 2004 DHS management directive emphasizes the need for an integrated acquisition organization and reiterates the Chief Procurement Officer's responsibility to manage, administer, and oversee all acquisition activity across DHS and to establish a qualified acquisition workforce. In practice, these responsibilities are spread throughout the department with unclear accountability. Further hampering efforts to effectively integrate the acquisition function, the directive provides that the Coast Guard and the Secret Service are statutorily exempt from its application. We found no reasonable basis to conclude that the directive could not be made applicable to them. The various organizations within DHS continue to operate in a largely disparate manner, with a lack of centralized oversight of compliance with the department's acquisition regulation and policies. Staffing disparities across the procurement organizations have only recently begun to be addressed. We found that staffing shortfalls led Procurement Operations to rely extensively on outside agencies for contracting support--often for a fee--and that this office lacked adequate internal controls to properly manage this interagency contracting activity. Because of the risks associated with interagency contracting, we recently designated this approach as a high-risk issue.[Footnote 11] Not Clear How Recent Management Directive Will Drive Integration of the Acquisition Organization: In October 2004, the Secretary of DHS signed a management directive entitled "Acquisition Line of Business Integration and Management." This directive, the department's principal guidance for "leading, governing, integrating, and managing" the acquisition function, states that DHS will standardize acquisition policies and procedures and continue to consolidate and integrate the number of systems supporting the acquisition function. It directs managers from each organization to commit resources to training, development, and certification of acquisition professionals. The directive also highlights the Chief Procurement Officer's broad authority, including: * management, administration, and oversight of departmentwide acquisition, financial assistance, strategic sourcing, and competitive sourcing programs; * promotion of career development and establishment of qualifications, training, and certification standards for the acquisition and financial assistance workforce; * development and publication of departmentwide acquisition and financial assistance regulations, directives, policies, and procedures; * designation of all heads of contracting activities; and: * development and maintenance of contracting officer warrant and financial assistance officer programs, including designation of qualified persons as contracting officers and financial assistance officers. However, the directive may not achieve its goal of creating an integrated acquisition organization, because it creates unclear working relationships between the Chief Procurement Officer and heads of DHS's principal organizations.[Footnote 12] For example, the Chief Procurement Officer and the director of Immigration and Customs Enforcement share responsibility for recruiting and selecting key acquisition officials, preparing performance ratings for the top manager of the contracting office, and providing appropriate resources to support the Chief Procurement Officer's initiatives. The policy leaves unclear how the responsibilities will be implemented or what enforcement authority the Chief Procurement Officer has to ensure that initiatives are carried out. In addition, directors are only required to "consider" the allocation of resources to meet procurement staffing levels in accordance with the Chief Procurement Officer's analysis. Agreements have not been developed on how the resources to train, develop, and certify acquisition professionals in the principal organizations will be identified or funded. In a relatively new department like DHS, which is still in the process of instituting procedures, developing a strong organizational culture, and establishing clear roles and missions, this concept of dual accountability--absent effective implementing guidance--may not accomplish the intended goals. The October 2004 management directive does not apply to the Coast Guard or the Secret Service, further hampering efforts to integrate the acquisition organization. The Coast Guard is one of the largest organizations within DHS, with obligations accounting for about $2.1 billion in fiscal year 2004, nearly 23 percent of the department's total. According to the directive, the Coast Guard is exempted by statute. We disagree. We are not aware of any explicit statutory exemption that would prevent the application of this directive. While several provisions in the Homeland Security Act would limit the range of management initiatives concerning the Coast Guard, none of them would appear to be applicable in this case. For example, the Homeland Security Act requires the Coast Guard to be maintained as a distinct entity within the department.[Footnote 13] Other limitations prevent the transfer of assets,[Footnote 14] alteration of missions,[Footnote 15] and changes in reporting relationships.[Footnote 16] The act also expressly provides that "the authorities, functions, and capabilities of the Coast Guard to perform its missions shall be maintained intact and without significant reduction after the transfer of the Coast Guard to the department." We find nothing in the directive that contravenes these limitations. Nothing in the document would reasonably appear to threaten the status of the Coast Guard as a distinct entity or otherwise impair its ability to perform statutory missions. We raised the question of statutory exemption with the DHS General Counsel, who shared our assessment concerning the explicit statutory exemptions. He viewed the applicability of the management directive as a policy matter, noting that "the determination of whether the application of all or part of this [management directive] would impact the [Coast Guard's] ability to perform its mission is not a legal matter and is more appropriately made by DHS policy officials." We agree that DHS officials, with sufficient reasons, could make a policy decision that a particular management directive impacts the Coast Guard's ability to perform its missions. In this instance, however, we found no evidence that such a decision had been made. The directive also provides that the Secret Service is exempted by statute. As with the Coast Guard, we are unaware of any specific statutory exemption that would prevent the application of the directive. The Homeland Security Act requires the Secret Service to be maintained as a distinct entity within the department.[Footnote 17] The 2005 Homeland Security Appropriations Act reiterates this requirement and imposes additional limitations on altering reporting relationships.[Footnote 18] Given the nature of the management directive, we also conclude that there is no apparent reason to exempt the Secret Service from its application.[Footnote 19] In Environment of Largely Disparate Procurement Organizations, Oversight of Acquisition Activity Lacking: DHS's principal organizations are, to a large extent, still functioning much as they did in premerger days with regard to acquisition-related functions. Embedded within seven of the procurement organizations are, for the most part, the same contracting staffs that joined DHS from their former agencies. The eighth organization, Procurement Operations, created to meet the needs of the many DHS organizations that do not have colocated procurement support, has a direct reporting chain to the Chief Procurement Officer. Until recently, the Chief Procurement Officer, whom DHS's top leadership delegated with the key responsibility of ensuring compliance with the department's acquisition regulation and policies, had only two staff members to carry out this duty. Consequently, the department's acquisition oversight program relies extensively on self-assessments by personnel in each procurement organization. A component of the oversight program is a recent initiative to review DHS acquisition plans, according to dollar thresholds, and to perform on-site evaluations of each procurement organization. The fiscal year 2005 budget provided the Chief Procurement Officer with five additional staff, but it is too soon to tell whether this number will be adequate to effectively implement the oversight program. Further, it remains unclear what the result would be if an organization were found not to be in compliance with DHS's acquisition regulation and policies. Our prior work shows that in a highly functioning acquisition organization, the chief procurement officer is in a position to oversee compliance with acquisition policies and processes by implementing strong oversight mechanisms.[Footnote 20] Adequate oversight of acquisition activities across DHS is imperative, in light of the department's mission and the problems that have been reported by us and inspectors general for some of the large agencies and organizations within the department. These reports have highlighted the lack of important management controls for monitoring contractors and ensuring efficiencies and effectiveness in the acquisition process. For example, the Department of Homeland Security Inspector General reported that during its first year of operation, the Transportation Security Administration relied extensively on contractors to accomplish its mission. It also found that contracting officers wrote contracts without clearly defined deliverables, and on occasion, contractors themselves were permitted to determine requirements and define deliverables. As a result, the cost of those initial contracts ballooned.[Footnote 21] The Transportation Security Administration is in the process of devising policies and procedures that require adequate procurement planning, contract structure, and contract oversight. We have also reported that the former U.S. Immigration and Naturalization Service did not have the basic infrastructure--including oversight of procurement activities--to ensure that its contracting office was effective. DHS's Inspector General recently reported that the Federal Emergency Management Agency discovered it has not been reporting or tracking procurements handled by its field offices.[Footnote 22] Challenges to Effective Implementation of Acquisition Workforce Plan: In July 2003, we recommended that DHS develop a data-driven assessment of the department's acquisition personnel, resulting in a workforce plan that would identify the number, location, skills, and competencies of the workforce.[Footnote 23] DHS concurred with the recommendation and has drafted a plan, based on best practices, that defines the acquisition workforce, focuses on the need for continuous training, and implements a certification program for contracting officials, program managers, and contracting officers' technical representatives.[Footnote 24] However, the department faces challenges in implementing the plan. As part of its acquisition workforce planning efforts, the department has not conducted an assessment of whether contracting staff within DHS are appropriately distributed for the varying workloads in each procurement organization. Our analysis shows that some disparities may exist. We divided the obligated fiscal year 2004 dollars for each contracting office by the number of contracting staff. While this approach is limited in that it does not take into account the complexity of the acquisitions being performed, it can provide senior leadership with an indication of whether disparities may exist in the contracting workforce. Figure 3 shows the amount of contracting obligations per contracting staff within DHS contracting activities. Figure 3: Comparison of Fiscal Year 2004 Dollars Obligated per Contracting Staff within Each DHS Contracting Office: [See PDF for image] [End of figure] As of September 2004, Procurement Operations had only 19 contracting staff to support a number of DHS organizations that, taken together, accounted for about 21 percent, or almost $2 billion, of DHS's fiscal year 2004 obligations. That year, Procurement Operations contracting staff on the average handled $101 million per employee, whereas the contracting staff for the Federal Law Enforcement Training Center on the average handled close to $2.7 million per employee. Disparities such as this may indicate the need to assess the numbers of contracting staff across the department to determine whether imbalances exist and whether actions are needed to correct the imbalances. Another challenge to effectively implementing the acquisition workforce plan pertains to the lack of enforcement of DHS's certification program. While the plan calls for program managers responsible for acquisitions to be certified in accordance with DHS's established training and experience requirements, in practice, the means to enforce compliance is lacking. In January 2005, the Director for the Acquisition Workforce Program, within the Office of the Chief Procurement Officer, determined that only 22 percent of the identified programs in the department had program managers that had documented that they had the training and experience requirements for certification. While the Office of the Chief Procurement Officer has issued a management directive requiring program mangers to meet the department's certification and training requirements--and the number of certified program managers has been increasing--accountability for complying with the certification program rests with the principal organizations to whom the program managers report. At present, according to DHS officials, no mechanism is in place to ensure that program managers take the required training and obtain certification from the Chief Procurement Officer. Struggling with Staffing Shortfalls, Procurement Operations Has Relied on Interagency Agreements but Failed to Follow DHS Guidance in Doing So: Established almost 1 year after DHS was formed and tasked with providing contracting support to the department organizations that did not have their own contracting support, Procurement Operations has struggled to manage its almost $2 billion workload because of staffing shortfalls. Lacking in-house capability, Procurement Operations has turned extensively to interagency contracting, and we found that management controls were not in place to effectively oversee this activity. Interagency contracting occurs when a federal agency obtains supplies or services through another federal agency, either by placing orders on existing contracts that have already been awarded by the other agency, or by asking the other agency to award and administer contracts or issue and administer task orders on its behalf. Use of these contracts demands a high degree of business acumen and flexibility on the part of the federal acquisition workforce. We found that Procurement Operations had transferred almost 90 percent of its obligations to other federal agencies through interagency agreements in fiscal year 2004. For example, DHS transferred $12 million to the Department of the Interior's National Business Center to obtain contractor operations and maintenance services at the Plum Island Animal Disease Center. Interior charged DHS $62,000 for this assistance. While some of the interagency agreements were for contracting support, others were for program support, such as sending funds to Department of Energy laboratories for providing a threat and capability assessment. DHS has issued a management directive that sets forth a number of requirements meant to ensure that internal controls are in place when using interagency contracting. Based on a random sample of 136 interagency agreements between Procurement Operations and outside agencies, we found that the office had not complied with the requirements in the directive.[Footnote 25] We can project, for example, that in fiscal year 2004: * 94 percent of Procurement Operations' files did not document that the contracting staff had conducted the required analysis of alternatives to justify the decision to pay an outside agency for contracting support; * 47 percent of the files did not identify the contracting officer's technical representative, although this information is required to be in the files; * 35 percent of the files did not contain the required determination and findings;[Footnote 26] and: * 96 percent of the files lacked an indication that contractor oversight had been performed. Further, we found that Procurement Operations was not tracking how much it is paying in fees to other agencies for contracting support. On the basis of our sample, we found that the office had spent $12.9 million in fees in fiscal year 2004. While the oversight problems we identified are in large part due to the staffing shortages in the office, we also found evidence that contracting staff lacked basic information on how to use interagency contracting. For example, a memo from Procurement Operations to DHS's Director of Acquisition Policy and Oversight requested clarification on what documentation is required in order to use another agency for contracting support. In September 2004, DHS's Office of General Counsel reviewed 20 of Procurement Operations' interagency agreements and found that 16 were not legally sufficient. For example, 13 agreements appeared to require performance by a contractor, but appropriate documentation was not included in the contract files. Three were insufficient because it was unclear whether the servicing agency would perform cost/price analysis and trade-off, or whether Procurement Operations had conducted a technical competition and expected the servicing agency to award without a trade-off. According to a Procurement Operations official, a fiscal year 2005 initiative for the office is to establish a policy for processing interagency agreements that provides clear guidance to staff. Procurement Operations to Use Working Capital Fund to Increase Staffing Levels: Since January 2004, Procurement Operations has increased its staffing level from 7 to 42 employees, and it plans to build to 127 staff in fiscal year 2005. Rather than use direct appropriations to fund the additional positions, the office plans to require the DHS organizations that rely on its contracting services to contribute to a working capital fund, to be replenished on a no-profit basis with payments based on the extent to which the various organizations use the office's support. Although the DHS budget for fiscal year 2005 includes $8.9 million to add contracting staff through the fund, we found that the mechanics of making the fund viable have not been worked out. Currently, Procurement Operations is negotiating agreements with each organization it supports to determine the terms and conditions of support and the dollar level to be contributed to the fund. According to DHS officials, this negotiation process has been problematic. For example, at the time of our review, the Science and Technology Directorate and Procurement Operations were having difficulty reaching a decision about who would be responsible for hiring the contracting staff that would support the directorate. As of January 2005, no agreements had been reached. Despite Adoption of Many Best Practices, Review Process for Major Investments Lacks Key Reviews and Some Management Controls: Some DHS organizations have large, complex, and high-cost acquisition programs that need to be closely managed. For example, the Bureau of Customs and Border Protection's Automated Commercial Environment system, which is a new trade processing system intended to improve the movement of goods imported into the United States, is projected to cost $5 billion, and the Coast Guard's Deepwater Program is expected to cost $17 billion and take 2 to 3 decades to complete.[Footnote 27] To review major, complex investments such as these (referred to as level 1 investments), DHS's Office of the Chief Financial Officer has put in place a multitiered process. DHS has taken positive steps in creating a knowledge-based framework, or philosophy, for managing major investments; however, it still lacks key reviews and deliverables--both best practices--within this framework to ensure that cost and schedule estimates for major investments are as accurate as possible. These reviews take place at critical junctures in the process and include demonstrating knowledge about technologies, design, and manufacturing processes. In addition, we found that contractor oversight, an important tool for managing programs, is not receiving high-level attention in the review process. Finally, we identified several areas of confusion surrounding the mechanics of implementing the process from a program manager's perspective. The management directive on the review process has been under revision for many months, and DHS officials could not tell us when the directive would be finalized. Key Elements of DHS's Investment Review Process: DHS's investment review process involves several different levels of review, depending on the dollar threshold and risk level of the program. The Investment Review Board makes decisions on level 1 investments with prior review and input from the Joint Requirements Council, which in turn seeks input from other DHS specialists who have expertise in such areas as asset management and information technology. In classifying investments as level 1, DHS considers the following criteria: contract costs; importance to DHS strategic and performance plans; high development, operating, or maintenance costs; high risk; high return; and significance in resource administration. Investments classified as levels 2, 3, or 4 are considered lower-level acquisitions and follow different investment review processes. In addition, many of the major DHS organizations, such as the Transportation Security Administration and the Coast Guard, have their own review processes, which occur prior to higher-level review in the department. Figure 4 illustrates who is involved in the decision-making process and the levels of review. Figure 4: General Depiction of DHS's Investment Review Process for Major, Complex Investments: [See PDF for image] [End of figure] Although Policy Governing the Review Process Has Established a Solid Risk Management Framework, Additional Best Practices Would Benefit DHS: DHS has adopted several best practices from lessons learned from leading commercial companies and successful federal programs that, if applied consistently, could refine its ability to reduce risk to meet cost and delivery targets for major investments. One of the best practices is a knowledge-based approach, or framework, for managers to hold reviews at key decision points in order to reduce risk before investing resources in the next phase of a program's development. The investment review policy provides guidance to program managers to provide knowledge about important aspects of a product at key points in the acquisition process and encourages them to reduce technology risk through demonstration prior to beginning a project. The policy also encourages program managers to demonstrate a product's design with critical design reviews and reduce manufacturing risk prior to a production decision. However, we found, based on our extensive body of work on this knowledge-based approach, that additional program reviews and knowledge deliverables could be incorporated into the process as internal controls to better position DHS to make well-informed decisions on its major, complex investments. Figure 5 generally depicts the major phases of the knowledge-based approach and the positioning of three knowledge points, or gates, when key reviews are scheduled. The figure applies the knowledge-based approach to DHS's investment review framework and displays an exclamation mark where key reviews or information are missing. Figure 5: Applying the Knowledge-Based Approach to DHS's Acquisition Framework: [See PDF for image] [End of figure] As shown in figure 5, DHS review points do not fully align with the knowledge-based approach. For example, DHS does not require a review to ensure that an investment's design performs as expected before investing in a prototype. In addition, DHS's mandatory review to proceed to production does not occur until after low-rate initial production is well under way. DHS does have a review for low-rate initial production; however, it is at the discretion of the Investment Review Board. Our past work has shown that successful investments reduce risk by ensuring that high levels of knowledge are achieved at these key points of development. We found that investments that were not reviewed at the appropriate points faced problems--such as redesign--that resulted in cost increases and schedule delays. We also found that some critical information is not addressed in DHS's investment review policy or the guidance provided to program managers. In other cases, it is made optional. For example, before program start (knowledge point 1) is approved, DHS policy requires program managers to identify an acquisition's key performance requirements and to have technical solutions in place. This information is then used to form cost and schedule estimates for the product's development to ensure that a match exists between requirements and resources. However, DHS policy does not establish cost and schedule estimates for the acquisition based on knowledge from preliminary designs. At knowledge point 2, while DHS policy requires program managers to identify and resolve critical operational issues before proceeding to production, initial reviews--such as the system and subsystem reviews--are not mandatory. Not all investments require the use of every piece of information included under a knowledge-based approach. Many of DHS's major investments use commercial, off-the-shelf products that do not require the same level of review as a complex, developmental investment would. However, DHS is investing in a number of major, complex systems, such as the Coast Guard's Deepwater Program and the U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT),[Footnote 28] which incorporate new technology and therefore require greater adherence to the knowledge-based approach in order to ensure risk is reduced before committing to the next phase of the investment. In addition, the added reviews and information included in the knowledge-based approach may still be required for programs that use existing technology, such as the Counter-MANPADS program, which involves placing military technology on commercial aircraft.[Footnote 29] In addition to the knowledge-based approach and its associated controls, DHS's investment review policy and guidance adopt a number of other important acquisition management practices, such as requiring program managers to submit acquisition plans and project management plans. However, a key practice, contractor tracking and oversight, is not fully incorporated in the policy and guidance. We have cited the need for increased contractor tracking and oversight for several large DHS programs. For example, we previously reported that the Coast Guard's Deepwater program needed increased management and contractor oversight. One activity of contract tracking and oversight requires that a quantitative set of software and system metrics are used to define and measure product quality and contractor performance. The Coast Guard had not developed measurable performance goals or adhered to effective procedures for holding the contractor accountable for its ongoing performance. In addition, we previously recommended that the US-VISIT program should improve attention to implementing acquisition best practices. While DHS agreed, and the US-VISIT program office has assigned responsibility for implementing the recommended management controls, the department has not yet developed explicit plans or time frames for defining and implementing acquisition best practices. A list of selected acquisition management practices and required activities is in appendix V. As Review Process Matures, Some Mechanics Still to Be Worked Out: The investment review process has been under revision for many months. According to DHS officials, the changes will include shifting responsibilities of some tiers in the review process and increasing the dollar threshold for level 1 investments. To date, the new process has not been finalized, and officials could not provide us with a time frame for completion. In the meantime, we found unclear guidance and confusion about several aspects of the process. In some cases, the confusion has resulted in key stakeholders, such as the Chief Procurement Officer, not receiving materials in time to conduct a thorough review and provide meaningful feedback prior to investment review meetings. The issues we found include the following: * Program managers have been provided with only draft guidance regarding the information they are required to submit and the time frames for submissions. This draft guidance is, in some cases, unclear. * Some DHS officials noted that their submissions to the review board had been rejected on an inconsistent basis with no explanation. * Program managers have not received formal training on the investment review process. Officials told us that some program managers have been unaware of when to submit information about their programs for review. * In practice, major investments in services are exempt from the review process and are only reviewed when done as part of a capital investment. Officials from the Office of the Chief Financial Officer who are in charge of the process told us that services investments are reviewed only when they are part of a capital investment because these acquisitions are not complex and therefore do not need the same level of scrutiny reserved for the acquisition of goods. Currently, program managers receive assistance in developing their review board submissions from a small number of staff in the Chief Financial Officer's Office of Program Analysis and Evaluation at the beginning of a program and at DHS's key decision points. However, because of limited resources, the office has only been able to provide limited support to programs to assist them in completing their submissions. Conclusions: In the 2 years since its merger, DHS has taken strides toward putting in place an acquisition organization that contains many promising elements, but the steps taken so far are not enough to ensure that the department is effectively managing the acquisition of the multitude of goods and services it needs to meet its mission. More needs to be done to fully integrate the department's acquisition function, to pave the way for the Chief Procurement Officer to fully carry out his responsibilities in a modern-day acquisition organization, and to put in place the strong internal controls needed to effectively manage interagency contracting activity and large, complex investments. Unless DHS's top leaders address these challenges, the department is at risk of continuing to exist with a fragmented acquisition organization that provides stopgap, ad hoc solutions. DHS has an opportunity, while it is still involved in transformational efforts, to avoid the complications that plague acquisition efforts in other long-established federal departments. Recommendations for Executive Action: To help ensure that DHS receives the goods and services it needs at the best value to the government, we recommend that the Secretary of Homeland Security take the following six actions: * establish a structure to ensure continued support for commodity councils, such as appointing full-time dedicated commodity managers, to ensure that the commodity councils develop long-term strategies, maintain momentum, and continue to realize savings; * provide the Office of the Chief Procurement Officer with sufficient resources and enforcement authority to enable effective, departmentwide oversight of acquisition policies and procedures; * conduct a departmentwide assessment of the number of contracting staff and, if a workload imbalance is found, take steps to correct it by re-aligning resources; * direct higher-level management attention to the implementation of the working capital fund (which is to be used to fund contracting staff for the Office of Procurement Operations) by, for example, determining the level of contracting support needed by the organizations relying on this office, ensuring that appropriate funds are committed to hire needed contracting staff, and ensuring that funds are available on an ongoing basis for continuity; * revise the October 2004 management directive "Acquisition Line of Business Integration and Management" to eliminate reference to the Coast Guard and Secret Service being statutorily exempt from complying; and: * ensure that DHS's management directive on interagency agreements is followed and that fees paid to other agencies are tracked. To help ensure that DHS leadership is aware of risks as they arise during the acquisition of major, complex systems, we recommend that the Secretary of Homeland Security take the following seven actions: * in making revisions to the investment review policy: * require for all complex, developmental investments a formal design review between the integration and demonstration of a program to ensure that the design is stable and has been demonstrated through prototype testing; * require for all complex, developmental investments a review before initial production; * require that program managers supply additional information--such as cost and schedule estimates based on results of a preliminary design review and critical design review--when their major, complex programs are reviewed; and: * require program managers to specifically address contractor oversight in their submissions to investment review boards. * ensure that stakeholders, including acquisition officials in the Office of the Chief Procurement Officer, have adequate time to review investment submissions and provide formal input to decision-making review boards; * implement training for program managers on the investment review process that emphasizes the importance of a knowledge-based approach; and: * require that major acquisitions of services be subject to oversight by the investment review board. Agency Comments and Our Evaluation: We provided a draft of this report to DHS for review and comment. In written comments, DHS generally agreed with our facts and conclusions and concurred with all of our recommendations. Regarding three of our recommendations on the investment review process, DHS stated that the actions exist in current directives and are already being done. We disagree. Our work demonstrated that DHS's review points do not fully align with the knowledge-based approach. For example, DHS's mandatory review to proceed into production does not occur until after low-rate initial production is well under way, and the review for starting low- rate initial production only occurs at the discretion of the Investment Review Board. Also, DHS's framework lacks the knowledge deliverables necessary at each key review to ensure that cost and schedule estimates for major investments are as predictable as possible. Our past work has shown that investments that were not reviewed at the appropriate points faced problems that resulted in cost increases and schedule delays. The department's comments are reprinted in appendix II. As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. We are sending copies of this report to other interested congressional committees, the Secretary of the Department of Homeland Security, and the Director of the Office of Management and Budget. In addition, the report will be available on GAO's Web site at http://www.gao.gov. If you or your staff have any questions concerning this report, please contact me at (937) 258-7915. Staff making major contributions to this report are listed in appendix VI. Signed by: Michael J. Sullivan, Director, Acquisition and Sourcing Management: [End of section] Appendix I: Scope and Methodology: To identify the areas where the Department of Homeland Security (DHS) has been successful in promoting collaboration among its various organizations, we interviewed senior acquisition officials at DHS headquarters and analyzed pertinent documents. We obtained information from the Office of Strategic Sourcing and Acquisition Systems to analyze how DHS has used strategic sourcing to leverage the department's buying power. We interviewed senior strategic sourcing officials at DHS headquarters to obtain information on how they identified the commodities with potential for savings. We reviewed studies, policies, guidance, and other documents related to ongoing or proposed strategic sourcing initiatives that leveraged buying power, cut costs, or achieved other performance benefits. Our review found that the department's strategic sourcing cost savings methodologies appear reasonable; however, we did not verify the accuracy of any strategic procurement costs savings reported to us. We asked senior strategic sourcing and commodity council participants about potential barriers to employing strategic sourcing at DHS. We interviewed the Director, Office of Small and Disadvantaged Business Utilization, and procurement officials within the department. We obtained policies, memorandums, and other documents from the small business office, procurement organizations, and the U.S. Small Business Administration. To determine areas where DHS faces challenges in integrating the acquisition function, we reviewed DHS organizational charts to gain insight into where the procurement offices fall in the hierarchy and to determine the lines of responsibility and authority between the various stakeholders in the acquisition process. We reviewed DHS policies, guidance, and procedures governing acquisition and analyzed internal audit reports, when available, to determine if those policies and procedures were being followed. We obtained statistics from DHS's Office of the Chief Procurement Officer on the department's procurements and the acquisition workforce. We interviewed procurement policy officials at headquarters and conducted interviews with personnel in each of the procurement organizations. To assess DHS's effectiveness in managing its acquisition workforce, we interviewed contracting and human resource officials at DHS headquarters. We analyzed DHS's processes and procedures for certifying program managers, warranting contracting officers, and tracking the acquisition workforce training. Lastly, we reviewed previous GAO work regarding best acquisition practices for organizational alignment and oversight. Because the Office of Procurement Operations has, by far, the highest level of interagency contracting activity in the department, we examined the office's database of contracts to gain an understanding of the general scope of activity involving interagency agreements. We randomly selected and then reviewed 136 interagency agreement contract files. We interviewed officials responsible for the data to ensure that the system they use to track procurement activity was adequate for identifying the sample population. We reviewed the interagency agreement contract files to assess key aspects of the acquisition process--such as a signed determination and findings in accordance with the Federal Acquisition Regulation, acquisition planning, and interagency agreement contract administration, including contractor oversight. We held follow-on discussions with the Office of Procurement Operations to discuss discrepancies noted in the files. Further information on our methodology and sampling error rates is as follows: * In fiscal year 2004, the Office of Procurement Operations predominately obtained contracting services for its customers through the issuance of interagency agreements with other government agencies. We drew our sample from an original population of 984 agreements. Of the 150 items we initially sampled, the department deleted 14 of these agreements and withdrew them from the data. As a result, we adjusted the population of agreements to which we are making estimates to 892 agreements. All estimates are to these 892 agreements based upon a sample size of 136 agreements. In addition, 4 files were missing and were not able to be reviewed, and one agreement was a duplicate. This resulted in only 131 files being reviewed. As we conducted our reviews, the agency became better at recording the agreements that should be included in its database. Currently, Procurement Operations reports that 1,104 agreements are contained in its records and total $1.8 billion. * We reviewed the files and documents for these 131 agreements, using as our criteria the department's directive on interagency agreements. The following projections are made from our sample of 136 agreements. The estimates from a statistical sample are always subject to some uncertainty because the entire collection is not reviewed. This uncertainty is called sampling error. Tables 2 and 3 show the sampling errors for certain factors relating to the issuance of interagency agreements. Table 2: Sampling Error at the 95 Percent Confidence Level for the Sample of Interagency Agreements: Observation: Analysis of alternatives not conducted; Number: Estimate: 840; Number: Sampling error: 36; Percentage: Estimate: 94.1%; Percentage: Sampling error: 4. Observation: Contracting officer's technical representatives not identified; Number: Estimate: 420; Number: Sampling error: 76; Percentage: Estimate: 47.1%; Percentage: Sampling error: 8.5. Observation: Determination and findings not prepared; Number: Estimate: 308; Number: Sampling error: 72; Percentage: Estimate: 34.6%; Percentage: Sampling error: 8.1. Observation: No indication that contractor oversight performed; Number: Estimate: 853; Number: Sampling error: 31; Percentage: Estimate: 95.6%; Percentage: Sampling error: 3.5. Source: GAO (data and analysis). [End of table] Table 3: Sampling Error at the 95 Percent Confidence Level for the Sample of Interagency Agreements Fee Payments: Observation: Fees paid on use of interagency agreements. Number: Estimate: 433; Number: Sampling Error: 76; Dollars: Estimate: $12,991,079; Dollars: Sampling Error: $2,280,189. Source: GAO (data and analysis). [End of table] To assess the department's progress in implementing a review process for major, complex systems, we compared DHS's acquisition policies for major acquisitions to our knowledge-based approach. We used information from several of our prior reports that examine how commercial best practices can improve outcomes for acquisition programs. Specifically, we compared and contrasted DHS's investment review process with the best practices for commercial acquisitions identified in our past reports. Our analysis focused on whether DHS's policies contained the measurable criteria and management controls necessary for minimizing cost, schedule, and performance risks. To clarify the content of the investment review process, we met with various DHS officials from the Office of the Chief Procurement Officer and the Office of Program Evaluation and Analysis within the Office of the Chief Financial Officer. We also discussed the review process with officials from DHS procurement organizations and collected reports and analyzed available program data on the current status of major acquisitions being carried out by DHS. We conducted our review from March 2004 through February 2005 in accordance with generally accepted government auditing standards. [End of section] Appendix II: Comments from the Department of Homeland Security: U.S. Department of Homeland Security: Washington, DC 20528: March 23, 2005: Mr. Michael J. Sullivan: Director (Acting), Acquisition and Sourcing Management: Government Accountability Office: Washington, DC 20548: Dear Mr. Sullivan: Re: Draft Report GAO-05-179, Successes and Challenges in DHS's Efforts to Create an Effective Acquisition Organization. Thank you for the opportunity to review your draft report We appreciate the opportunity to comment on the draft report and the time your staff afforded us during their preparation of the report. We generally concur with the report and with the recommended executive actions listed at the conclusion of your report. Your report effectively discusses many of the integration, accountability and staffing issues confronting the Department. Any such report can only represent a snapshot in time. In that context, it is understandable that the report does not reflect the most current staffing information. For example, the report mentions that an increase in the acquisition oversight staff was authorized in the FY-05 budget. The oversight staff has three of the five billets filled as of this date and will complete the recruitment and hiring process in the next 60 days for the remaining billets. Similarly, the Headquarters Office of Procurement Operations has made strides in its staffing efforts and now has 42 personnel aboard, though much remains to be done in this area. DHS agrees that a rigorous oversight approach is central to an effective acquisition and investment review program and believes that the initiatives in those areas will address the establishment of an effective framework. These observations do not alter our view that the report provides valuable and useful information. The report addresses the Investment Review Process in depth. DHS has conducted a joint review of the departmental Investment Review Process in response to the direction from the Deputy Secretary on January 24, 2005. The actions required are consistent with the draft GAO report recommendations, and considered the GAO Statement of Facts dated December, 2004. Several actions have already been initiated to ensure departmental oversight without imposing burdensome and redundant processes on the DHS operating agencies. Below is the reply to each of the GAO's seven recommendations regarding investment review provided by Program Analysis & Evaluation. Note DHS will place further emphasis on these recommendations in all Department investment review process guidance. GAO Recommendation 1: Require major service contracts be reviewed by the IRB. Concur. MD 1400 is being updated for publishing this spring to reflect interim changes made to the DHS Investment Review process. The IRB will conduct annual reviews of large planned service contracts and support programs. This review will significantly improve oversight of large planned expenditures that are not capital investments. GAO Recommendation 2: Train Program Managers in the investment review process and guidance. Concur. The following initiatives are planned for implementation over the next year: Provide briefing and training on the MD 1400 revisions. Further, DHS is developing a well qualified program management workforce. The Undersecretary for Management is providing policy and training necessary to obtain and maintain program management competencies, knowledge and act on best practices. DHS directives require managers of level one acquisition projects to be DHS certified within one year of appointment. The department augments the certification program with work tools and best practices for program managers. GAO Recommendation 3: Require for all complex, developmental investment a formal design review between the integration and demonstration of a program to ensure that the design is stable and has been demonstrated through prototype testing. Concur. The recommended actions exist in current directives and are being done. We will, however, further emphasize the recommendation in the revisions to the Department's directives and guidance. GAO Recommendation 4: Require a review before initial production. Concur. See the response to Recommendation 3. GAO Recommendation 5: Require Program Managers to supply additional information-such as cost and schedule estimates based on results of a preliminary design review and critical design review. Concur. The recommended actions exist in current directives and are being done. We will, however, further emphasize the recommendation in the revisions to the Department's directives and guidance. GAO Recommendation 6: Require Program Managers to specifically address contractor oversight in their submission to IRB. Concur. Currently, DHS requires an Acquisition Program Baseline (APB) to gain better cost, schedule and performance visibility for each investment project. To further improve project oversight, the department will deploy an Earned Value Management System (EVMS). This initiative will require selection of a reporting tool that will include a set of dashboard gauges to monitor program adherence to cost, schedule and performance. The APB requires cost, schedule and performance information and from the projects, and the EVMS includes contractor oversight. At any time, the IRB may require further review of any program. GAO Recommendation 7: Give stakeholders, specifically DHS CPO, time to review submission prior to IRB and JRC. Concur. The DHS CPO is represented on both the JRC and on the IRB, and staffs will ensue adequate review time for principals. We thank you again for the opportunity to review the report and provide comments. Sincerely, Signed by: Steven J. Pecinovsky: Acting Director, Departmental GAO/OIG Liaison Office: [End of section] Appendix III: Office of Procurement Operations' Customers: Chief Financial Officer: Chief Human Capital Officer: Chief Information Officer: Chief Procurement Officer: Chief of Administrative Services: Chief of Staff: Citizenship and Immigration Services: Citizenship and Immigration Services Ombudsman: Civil Rights and Civil Liberties: Communications Director: Counter Narcotics Officer: Deputy Secretary: Executive Secretary: General Counsel: Headquarters Operational Integration Staff: Homeland Security Advisory Council: Homeland Security Operations Center: Immigration Statistics: Information Analysis and Infrastructure Protection: International Affairs: Office of the Secretary: Legislative Affairs: National Capital Region: Office of the Under Secretary--Border and Transportation Security: Press Secretary: Privacy Officer: Private Sector: Public Affairs: Science and Technology: Security: State and Local Government Coordination (includes Office of Domestic Preparedness): Under Secretary for Management: White House Liaison: [End of section] Appendix IV: Summary of Selected Commodity Council Strategic Sourcing Initiatives: Commodity council: Office Supplies; Strategic sourcing actions taken: Leverages DHS's purchasing power and reduces amount spent on office supplies: * For its main strategic sourcing initiative, this council partnered with the Department of Defense's Electronic Mall (DOD Emall). DOD Emall is an Internet-based catalog ordering system that provides 24-hour-a- day, 7-day-a-week access to over 200 small and large office supply vendors; * Because many military services also purchase office supplies through DOD Emall, DHS can take advantage of large volume discounts to increase its buying power; * As of June 1, 2004, DHS mandated all of its purchase cardholders to exclusively use DOD Emall to buy office supplies. As a result, DHS has already become the third-largest federal government user of DOD Emall. For the 4-month period starting June 1, 2004, DHS spent over $14.5 million on DOD Emall, representing 16.7 percent of the total dollars spent on Emall; * Before the full implementation of DOD Emall, the council also generated savings by using an office supplies blanket purchase agreement that the Transportation Security Administration had in place to provide office supplies to other DHS principal organizations; * Through September 30, 2004, the council estimated cost savings of $8 million. Commodity council: Boats; Strategic sourcing actions taken: Establishes standard boat procurements to decrease life cycle costs, facilitate interoperability, reduce training burden, and provide immediate cost savings: * For example, Customs and Border Patrol needed six boats in fiscal year 2004. Instead of creating a new contract, the council explored whether there were any existing contract vehicles within DHS that could satisfy the need; * The council purchased the six boats through an existing Coast Guard contract and took advantage of large volume discounts; * As a result of these strategic sourcing efforts, the council was able to acquire each boat for $50,000 less than anticipated, resulting in a total of $300,000 in cost savings. Commodity council: Weapons; Strategic sourcing actions taken: Identifies and consolidates emerging firearms and ammunition requirements for DHS: * In fiscal year 2004, the council planned to acquire pistols for DHS organizations to meet critical, mission-driven requirements; * The council surveyed all DHS organizations interested in weapons, factored in the end users' requirements, and established a list of potential vendors that could provide best value for pistols and satisfy requirements; * As a result, DHS awarded contracts to two vendors for 65,000 pistols each over 5 years--the largest pistol acquisition in the history of U.S. law enforcement; * As of September 20, 2004, the council has reported cost savings of over $4.1 million and $915,000 in cost avoidance. Commodity council: Energy; Strategic sourcing actions taken: Identifies strategies for aggregating and centralizing DHS's energy procurements to take advantage of economies of scale and negotiate more competitive prices with suppliers: * Although DHS spent approximately $58 million for electricity in fiscal year 2003, only $9 million of the spending was for negotiable energy services in states with deregulated electricity industries. Nevertheless, the council estimates more than $705,000 in cost savings through September 30, 2004. Source: DHS (data); GAO (presentation). [End of table] [End of section] Appendix V: Selected Acquisition Management Best Practices: Best practices: Acquisition planning; To ensure that reasonable planning for all parts of the acquisition is conducted; Activity: Plans are prepared during acquisition planning and maintained throughout the acquisition; Planning addresses the entire acquisition process, as well as life cycle support of the products being acquired; The acquisition organization has a written policy for planning the acquisition; Responsibility for acquisition planning activities is designated. Best practices: Contract tracking and oversight; To ensure that contract activities are performed in accordance with contractual requirements; Activity: The acquiring organization has sufficient insight into the contractor's activities to manage and control the contractor and ensure that contract requirements are met; The acquiring organization and contractor maintain ongoing communication; commitments are agreed to and implemented by both parties; All contract changes are managed throughout the life of the contract; The acquisition organization has a written policy for contract tracking and oversight; Responsibility for contract tracking and oversight activities is designated; The acquiring organization involves contracting specialists in the execution of the contract; A quantitative set of software and system metrics is used to define and measure product quality and contractor performance; In addition to incentives for meeting cost and schedule estimates, measurable, metrics-based product quality incentives are explicitly stated in the contract. Best practices: Risk management; To ensure that risks are proactively identified and systematically mitigated; Activity: Projectwide participation in the identification and mitigation of risks is encouraged; The defined acquisition process provides for the identification, analysis, and mitigation of risks; Milestone reviews include the status of identified risks; The acquisition organization has a written policy for managing acquisition risk; Responsibility for acquisition risk management activities is designated. Source: GAO (data); GAO (presentation). Note: See GAO, Information Technology: DoD's Acquisition Policies and Guidance Need to Incorporate Additional Best Practices and Controls, GAO-04-722 (Washington, D.C.: July 30, 2004) for a complete list of the acquisition management practices and required activities. [End of table] [End of section] Appendix VI: GAO Contacts and Staff Acknowledgments: Contacts: Michael J. Sullivan, (937) 258-7915 Michele Mackin, (202) 512-4309: Staff Acknowledgments: In addition to those named above, Daniel Chen, Lily Chin, Benjamin Federlein, Arthur James Jr., John Krump, Jose Ramos, and Russell Reiter made key contributions to this report. FOOTNOTES [1] The President signed legislation to create DHS on November 25, 2002. Homeland Security Act of 2002, Pub. L. No. 107-296, 116 stat. 2135, Nov. 25, 2002. [2] When the department was established, 22 agencies and organizations were brought in; Plum Island Animal Disease Center joined DHS afterward. [3] GAO, High-Risk Series: An Update, GAO-03-119 (Washington, D.C.: January 2003). [4] Federal Acquisition Regulation 2.1, Definitions. [5] Strategic sourcing is a process used by leading commercial companies and a small number of federal agencies to establish an organizationwide approach to leveraging the organization's buying power and fostering new ways of doing business. [6] The Small Business Reauthorization Act of 1997 directed the President to establish a goal of not less than 23 percent of the federal government's prime contracting dollars to be awarded to small businesses each fiscal year. The Small Business Administration is charged with ensuring that federal agencies' goals, in the aggregate, meet or exceed the 23 percent goal. Pub. L. No. 105-135, 111 stat. 2592, Dec. 2, 1997. [7] Spend analysis is a tool that organizations use to acquire knowledge about how much is being spent for what goods and services, who are the buyers, and who are the suppliers. GAO, Best Practices: Using Spend Analysis to Help Agencies Take a More Strategic Approach to Procurement, GAO-04-870 (Washington, D.C.: Sept. 16, 2004). [8] Our review found that the department's strategic sourcing cost savings methodologies appear reasonable. However, we did not verify the accuracy of any strategic procurement cost savings reported to us. [9] GAO, Best Practices: Taking a Strategic Approach Could Improve DOD's Acquisition of Services, GAO-02-230 (Washington, D.C.: Jan. 18, 2002). [10] DOD Emall is an Internet-based marketplace that allows DOD and other federal purchasers to access DOD's wide variety of vendors and catalogs and acquire off-the-shelf items from the commercial marketplace. [11] GAO, High-Risk Series: An Update, GAO-05-207 (January 2005, Washington, D.C.) [12] Two prior GAO reports, Transportation Security Administration: High-Level Attention Needed to Strengthen Acquisition Function, GAO-04-544 (Washington, D.C.: May 28, 2004), and Contract Management: INS Contracting Weaknesses Need Attention from the Department of Homeland Security, GAO-03-799 (Washington, D.C.: July 25, 2003), address the importance of having clear lines of authority to ensure that contracting activity is effective and enables the department to get best value on goods and services. [13] Pub. L. No. 107-296, § 888(b) November 25, 2002. [14] Id. at § 888(d). [15] Pub. L. No. 107-296, § 888(e), November 25, 2002. [16] Id. at §§104 and 888(g). [17] Pub. L. 107-296, § 521, November 25, 2002. [18] Pub. L. 108-334, § 528, October 18, 2004. "None of the funds available in this Act shall be available to maintain the United States Secret Service as anything but a distinct entity within the Department of Homeland Security and shall not be used to merge the United States Secret Service with any other department function, cause any personnel and operational elements of the United States Secret Service to report to an individual other than the Director of the United States Secret Service, or cause the Director to report directly to any individual other than the Secretary of Homeland Security." Department of Homeland Security Appropriations Act, 2005, Pub. L. No. 108-334, § 521, 118 stat. 1298, Oct. 18, 2004. [19] Given the similarity of the issue pertaining to the Coast Guard, we did not separately seek the view of the DHS General Counsel concerning the exemption of the Secret Service. [20] GAO-02-230 and GAO-04-544. [21] DHS Office of the Inspector General, Semiannual Report to the Congress, (Washington, D.C.: April 30, 2003), and Review of the Status of Department of Homeland Security Efforts to Address Its Major Management Challenges, OIG-04-21 (Washington, D.C.: March 2004). [22] OIG-04-21. [23] GAO-03-799. [24] Contracting officers' technical representatives represent the contracting officer in monitoring the contractor's performance. [25] See appendix I for details on our methodology and sampling error rates. [26] Determination and findings is a written approval by an authorized official to take certain contract actions. The determination is a conclusion or decision supported by the findings. The findings are statements of fact or rationale essential to support the determination. [27] At the time of our review, the Coast Guard had revised its requirements for the Deepwater program based on the new homeland security mission. We did not have information on the new cost of the program. [28] US-VISIT is a governmentwide program to collect, maintain, and share information on foreign nationals. The program's goals are to enhance national security, facilitate legitimate trade and travel, contribute to the integrity of the U.S. immigration system, and adhere to U.S. privacy laws and policies. [29] Counter-MANPADS (man-portable air defense systems) is a protective system for U.S. commercial aircraft against shoulder-fired missiles. We recently issued a report that recommended additional information be provided at key decision points. See GAO, The Department of Homeland Security Needs to Fully Adopt a Knowledge-based Approach to Its Counter-MANPADS Development Program, GAO-04-341R (Washington, D.C.: Jan. 30, 2004). GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.