Business Systems Modernization

Strategy for Evolving DOD's Business Enterprise Architecture Offers a Conceptual Approach, but Execution Details Are Needed Gao ID: GAO-07-451 April 16, 2007

In 1995, we first designated the Department of Defense's (DOD) business systems modernization program as "high risk," and we continue to designate it as such today. To assist in addressing this high-risk area, Congress passed legislation consistent with prior GAO recommendations for Defense to develop a business enterprise architecture (BEA). In September 2006, DOD released version 4.0 of its BEA, which despite improvements over prior versions, was not aligned with component architectures. Subsequently, Defense issued a strategy for extending its BEA to the component military services and defense agencies. To support GAO's legislative mandate to review DOD's BEA, GAO assessed DOD's progress in defining this strategy by comparing it with prior findings and recommendations relevant to the strategy's content.

DOD's Business Mission Area federation strategy for extending its BEA to the military departments and defense agencies provides a foundation on which to build and align the department's parent business architecture (the BEA) with its subordinate architectures (i.e., component- and program-level architectures). In particular, the strategy, which was released in September 2006, states the department's federated architecture goals; describes federation concepts that are to be applied; and explains high-level activities, capabilities, products, and services that are intended to facilitate implementation of the concepts. However, the strategy does not adequately define the tasks needed to achieve the strategy's goals, including those associated with executing high-level activities and providing related capabilities, products, and services. Specifically, it does not adequately address how strategy execution will be governed, including assignment of roles and responsibilities, measurement of progress and results, and provision of resources. Also, the strategy does not address, among other things, how the component architectures will be aligned with the latest version of the BEA and how it will identify and provide for reuse of common applications and systems across the department. According to program officials, the department intends to develop more detailed plans to execute the strategy. This means that much remains to be decided and accomplished before DOD will have the means in place to create a federated BEA that satisfies GAO's prior recommendations and legislative requirements. Without one, the department will remain challenged in its ability to minimize duplication and maximize interoperability among its thousands of business systems.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-07-451, Business Systems Modernization: Strategy for Evolving DOD's Business Enterprise Architecture Offers a Conceptual Approach, but Execution Details Are Needed This is the accessible text file for GAO report number GAO-07-451 entitled 'Business Systems Modernization: Strategy for Evolving DOD‘s Business Enterprise Architecture Offers a Conceptual Approach, but Execution Details Are Needed' which was released on April 16, 2007. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Committees: April 2007: Business Systems Modernization: Strategy for Evolving DOD‘s Business Enterprise Architecture Offers a Conceptual Approach, but Execution Details Are Needed. GAO-07-451: GAO Highlights: Highlights of GAO-07-451, a report to congressional committees. Why GAO Did This Study: In 1995, we first designated the Department of Defense‘s (DOD) business systems modernization program as ’high risk,“ and we continue to designate it as such today. To assist in addressing this high-risk area, Congress passed legislation consistent with prior GAO recommendations for Defense to develop a business enterprise architecture (BEA). In September 2006, DOD released version 4.0 of its BEA, which despite improvements over prior versions, was not aligned with component architectures. Subsequently, Defense issued a strategy for extending its BEA to the component military services and defense agencies. To support GAO‘s legislative mandate to review DOD‘s BEA, GAO assessed DOD‘s progress in defining this strategy by comparing it with prior findings and recommendations relevant to the strategy‘s content. What GAO Found: DOD‘s Business Mission Area federation strategy for extending its BEA to the military departments and defense agencies provides a foundation on which to build and align the department‘s parent business architecture (the BEA) with its subordinate architectures (i.e., component- and program-level architectures). (See figure.) In particular, the strategy, which was released in September 2006, states the department‘s federated architecture goals; describes federation concepts that are to be applied; and explains high-level activities, capabilities, products, and services that are intended to facilitate implementation of the concepts. Figure: Simplified Diagram of DOD‘s Business Mission Area Federated Architecture. [See PDF for image] Source: GAO analysis of DOD data. [End of figure] However, the strategy does not adequately define the tasks needed to achieve the strategy‘s goals, including those associated with executing high-level activities and providing related capabilities, products, and services. Specifically, it does not adequately address how strategy execution will be governed, including assignment of roles and responsibilities, measurement of progress and results, and provision of resources. Also, the strategy does not address, among other things, how the component architectures will be aligned with the latest version of the BEA and how it will identify and provide for reuse of common applications and systems across the department. According to program officials, the department intends to develop more detailed plans to execute the strategy. This means that much remains to be decided and accomplished before DOD will have the means in place to create a federated BEA that satisfies GAO‘s prior recommendations and legislative requirements. Without one, the department will remain challenged in its ability to minimize duplication and maximize interoperability among its thousands of business systems. What GAO Recommends: To assist DOD in its efforts to evolve and extend its BEA, GAO is augmenting a prior recommendation to the Secretary of Defense for developing an architecture development management plan by recommending that this plan incorporate details needed to execute DOD‘s Business Mission Area federation strategy. In comments, DOD largely disagreed with GAO‘s recommendation, noting that elements of it were either unnecessary or not appropriately focused. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-451]. To view the full product, including the scope and methodology, click on the link above. For more information, contact Randolph C. Hite at (202) 512-3439 or hiter@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: DOD Is in the Early Stages of Federating Its BEA and Much Remains to Be Decided and Accomplished: Conclusions: Recommendation for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objective, Scope, and Methodology: Appendix II: Comments from the Department of Defense: Appendix III: GAO Contact and Staff Acknowledgments: Tables: Table 1: Number of Framework Elements That Were Fully, Partially, and Not Satisfied by the Military Departments: Table 2: High-level Steps for Achieving Operational and Systems View Federation: Figures: Figure 1: Simplified Depiction of the DOD Enterprise Architecture Federation Strategy: Figure 2: Simplified Diagram of DOD‘s Business Mission Area Federated Architecture: Abbreviations: ASD(NII)/CIO: Assistant Secretary of Defense (Networks and Information Integration)/Chief Information Officer: BEA: business enterprise architecture: BMA: Business Mission Area: BTA: Business Transformation Agency: CIO: Chief Information Officer: DBSMC: Defense Business Systems Management Committee: DISA: Defense Information Systems Agency: DLA: Defense Logistics Agency: DOD: Department of Defense: EA: enterprise architecture: ETP: enterprise transition plan: IT: information technology: OMB: Office of Management and Budget: SOA: service-oriented architecture: [End of section] United States Government Accountability Office: Washington, DC 20548: April 16, 2007: Congressional Committees: For decades, the Department of Defense (DOD) has been challenged in repeated attempts to modernize its timeworn business systems. [Footnote 1] In 1995, we designated DOD‘s business systems modernization program as ’high risk,“ and we continue to designate it as such today. [Footnote 2] As our research on public and private sector organizations has shown, one essential ingredient to a successful systems modernization program is having and using a well-defined enterprise architecture (EA). [Footnote 3] Accordingly, we made recommendations to the Secretary of Defense in May 2001 that included the means for effectively developing an EA. [Footnote 4] In July 2001, the department initiated a business management modernization program to, among other things, develop one. Between 2001 and 2005, we reported that the department‘s business management modernization program was not being effectively managed, concluding in 2005 that hundreds of millions of dollars had been spent on a business enterprise architecture (BEA) that had limited use. [Footnote 5] Accordingly, we made additional architecture-related recommendations. To assist DOD in addressing its modernization management challenges, Congress included provisions in the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 [Footnote 6] that were consistent with our recommendations, including those for developing a BEA and associated enterprise transition plan (ETP). In 2006, [Footnote 7] we reported that, among other things, DOD had made important progress in developing its BEA, but that much remained to be accomplished relative to the act‘s requirements and relevant guidance. For example, we reported that the BEA was not adequately linked to the military departments and defense agency architectures. To address these shortfalls, DOD issued a strategy for ’federating“ or extending its architecture. To support GAO‘s legislative mandate to review DOD‘s BEA and as agreed with your offices, the objective of this review was to determine DOD‘s progress in defining its Business Mission Area (BMA) federation strategy. To accomplish our objective, we compared the federation strategy and associated plans with prior findings and recommendations relative to the content of the strategy, and interviewed DOD officials regarding the strategy‘s executability. We performed our work at DOD headquarters in Arlington, Virginia, from August 2006 through March 2007 in accordance with generally accepted government auditing standards. Additional details on our objective, scope, and methodology are contained in appendix I. Results in Brief: DOD‘s BMA federation strategy, which was released in September 2006, provides a foundation on which to build and align the department‘s parent business architecture (the BEA) with its subordinate architectures (i.e., component- and program-level architectures). In particular, this strategy states the department‘s federated architecture goals; describes federation concepts that are to be applied; and explains high-level activities, capabilities, products, and services that are intended to facilitate implementation of the concepts. However, the strategy does not adequately define the tasks needed to achieve the strategy‘s goals, including those associated with executing high-level activities and providing related capabilities, products, and services. Specifically, it does not adequately address how strategy execution will be governed, including assignment of roles and responsibilities, measurement of progress and results, and provision of resources. In addition, the strategy does not clearly describe the relationships, dependencies, and touch points with other federation strategies. Also, the strategy does not address, among other things, how the architectures of the military departments will align with the latest version of the BEA and how DOD will identify and provide for sharing of common applications and systems across the department. Moreover, the strategy does not include milestones for executing the activities and related capabilities, products, and services. According to program officials, the department is in the early stages of defining and implementing its strategy and intends to develop more detailed plans. As a result, much remains to be decided and accomplished before DOD will have in place the means to create a federated architecture, and thus be able to satisfy both our prior recommendations and legislative requirements aimed at adopting an architecture-centric approach to departmentwide business systems investment management. To assist DOD in its efforts to evolve and extend its BEA, we are augmenting our prior recommendation to the Secretary of Defense to develop an integrated architecture development management plan [Footnote 8] by recommending that this plan provide for effective execution of its BMA federation strategy. In written comments on a draft of this report, signed by the DOD Deputy Chief Information Officer and the Deputy Under Secretary of Defense (Business Transformation) and reprinted in appendix II, the department largely disagreed with our recommendation for two primary reasons. First, the department stated that three of the five elements in our recommendation have either already been addressed through policy or are embedded in a prior recommendation. Specifically, it said that the element relating to ensuring alignment with other federation strategies is not needed because the DOD EA federation strategy is the single architecture federation strategy for the department and the other architecture federation strategies supplement this overarching strategy. We do not question the department‘s comment about the relationships among the strategies. However, we believe that this element of our recommendation is needed because its intent is to recognize these relationships by promoting collaboration and ensuring linkages among the various strategies. DOD also stated that the element of our recommendation relating to component architecture alignment with incremental versions of the BEA has been implemented both in policy and execution to comply with legislative requirements, to include DOD‘s development and use of the Architecture Compliance and Requirements Traceability tool. We disagree. While we recognize DOD‘s efforts to align programs to the BEA, our recommendation focuses on the lack of a discussion in the BMA federation strategy on how component architectures will be linked to the BEA, including the lack of component architecture alignment guidance, criteria, and tools. Lastly, DOD stated that the element of our recommendation relating to milestones for gauging progress is and will continue to be monitored in the department‘s enterprise transition plan. While we have previously recognized that the transition plan provides information on the progress of major investments over the last 6 months”including key accomplishments and milestones attained, this element of our recommendation is intended to address the lack of measures (e.g., return on investment of service reuse) or specific completion dates for the activities and related capabilities, products, and services associated with the BMA federation strategy. Second, the department stated that while it agreed with the core intent of the remaining two elements of our recommendation, these elements should be sufficiently specific to permit reasonable implementation and should be directed to the appropriate parties within the department. Our recommendation is not intended to dictate who should develop the policies or guidance for managing architectures within a federated environment. Rather it is focused on developing plans that describe how the BMA will adopt and implement the policies and guidance relating to federation governance and service orientation. To further ensure that our recommendation is properly interpreted and implemented and to address DOD‘s comments about directing the recommendation to the appropriate parties, we have slightly modified it. Background: DOD is a massive and complex organization. To illustrate, the department reported that its fiscal year 2006 operations involved approximately $1.4 trillion in assets and $2.0 trillion in liabilities; more than 2.9 million in military and civilian personnel; and $581 billion in net cost of operations. Organizationally, the department includes the Office of the Secretary of Defense, the Chairman of the Joint Chiefs of Staff, the military departments, numerous defense agencies and field activities, and various unified combatant commands that are responsible for either specific geographic regions or specific functions. In support of its military operations, the department performs an assortment of interrelated and interdependent business functions, including logistics management, procurement, health care management, and financial management. As we have previously reported, [Footnote 9] the DOD systems environment that supports these business functions is overly complex and error-prone, and is characterized by (1) little standardization across the department, (2) multiple systems performing the same tasks, (3) the same data stored in multiple systems, and (4) the need for data to be entered manually into multiple systems. Moreover, DOD recently reported that this systems environment is comprised of approximately 3,100 separate business systems. For fiscal year 2006, Congress appropriated approximately $15.5 billion to DOD, and for fiscal year 2007, DOD has requested about $16 billion in appropriated funds to operate, maintain, and modernize these business systems and associated infrastructure. As we have previously reported, [Footnote 10] the department‘s nonintegrated and duplicative systems contribute to fraud, waste, and abuse. In fact, DOD currently bears responsibility, in whole or in part, for 15 of our 27 high-risk areas. [Footnote 11] Eight of these areas are specific to DOD [Footnote 12] and the department shares responsibility for 7 other governmentwide high-risk areas. [Footnote 13] DOD‘s business systems modernization is one of the high-risk areas, and it is an essential enabler to addressing many of the department‘s other high-risk areas. For example, modernized business systems are integral to the department‘s efforts to address its financial, supply chain, and information security management high-risk areas. Enterprise Architecture Is Critical to Achieving Successful Business Systems Modernization: Effective use of an enterprise architecture”a modernization blueprint”is a hallmark of successful public and private organizations. For more than a decade, we have promoted the use of architectures to guide and constrain systems modernization, recognizing them as a crucial means to this challenging goal: optimally defined operational and technological environments. Congress, the Office of Management and Budget (OMB), and the federal Chief Information Officer‘s (CIO) Council have also recognized the importance of an architecture-centric approach to modernization. The Clinger-Cohen Act of 1996 [Footnote 14] mandates that an agency‘s CIO develop, maintain, and facilitate the implementation of an information technology (IT) architecture. Furthermore, the E-Government Act of 2002 [Footnote 15] requires OMB to oversee the development of enterprise architectures within and across agencies. In addition, we, OMB, and the CIO Council have issued guidance that emphasizes the need for system investments to be consistent with these architectures. [Footnote 16] Enterprise Architecture: A Brief Description: An enterprise architecture provides a clear and comprehensive picture of an entity, whether it is an organization (e.g., a federal department) or a functional or mission area that cuts across more than one organization (e.g., financial management). This picture consists of snapshots of both the enterprise‘s current (’As Is“) environment and its target (’To Be“) environment. These snapshots consist of ’views,“ which are one or more interdependent and interrelated architecture products (e.g., models, diagrams, matrices, and text) that provide logical or technical representations of the enterprise. The architecture also includes a transition or sequencing plan, which is based on an analysis of the gaps between the ’As Is“ and ’To Be“ environments. This plan provides a temporal road map for moving between the two environments and incorporates such considerations as technology opportunities, marketplace trends, fiscal and budgetary constraints, institutional system development and acquisition capabilities, legacy and new system dependencies and life expectancies, and the projected value of competing investments. The suite of products produced for a given entity‘s enterprise architecture, including its structure and content, is largely governed by the framework used to develop the architecture. Since the 1980s, various architecture frameworks have been developed, such as John A. Zachman‘s ’A Framework for Information Systems Architecture“ [Footnote 17] and the DOD Architecture Framework. [Footnote 18] The importance of developing, implementing, and maintaining an enterprise architecture is a basic tenet of both organizational transformation and systems modernization. Managed properly, an enterprise architecture can clarify and help optimize the interdependencies and relationships among an organization‘s business operations (and the underlying IT infrastructure and applications) that support these operations. Moreover, when an enterprise architecture is employed in concert with other important management controls, such as portfolio-based capital planning and investment control practices, architectures can greatly increase the chances that an organization‘s operational and IT environments will be configured to optimize mission performance. Our experience with federal agencies has shown that investing in IT without defining these investments in the context of an architecture often results in systems that are duplicative, not well integrated, and unnecessarily costly to maintain and interface. [Footnote 19] One approach to structuring an enterprise architecture is referred to as a federated enterprise architecture. Such a structure treats the architecture as a family of coherent but distinct member architectures that conform to an overarching architectural view and rule set. This approach recognizes that each member of the federation has unique goals and needs as well as common roles and responsibilities with the levels above and below it. Under a federated approach, member architectures are substantially autonomous, although they also inherit certain rules, policies, procedures, and services from higher-level architectures. As such, a federated architecture enables component organization autonomy, while ensuring enterprisewide linkages and alignment where appropriate. Where commonality among components exists, there are also opportunities for identifying and leveraging shared services. A service-oriented architecture (SOA) is an approach for sharing business capabilities across the enterprise by designing functions and applications as discrete, reusable, and business-oriented services. As such, service orientation permits sharing capabilities that may be under the control of different component organizations. As we have previously reported, [Footnote 20] such capabilities or services need to be, among other things, (1) self-contained, meaning that they do not depend on any other functions or applications to execute a discrete unit of work; (2) published and exposed as self-describing business capabilities that can be accessed and used; and (3) subscribed to via well-defined and standardized interfaces. A SOA approach is thus not only intended to reduce redundancy and increase integration, but also to provide the kind of flexibility needed to support a quicker response to changing and evolving business requirements and emerging conditions. DOD‘s Efforts to Adopt a Federated Approach to Architecting All of Its Mission Areas: The Office of the Assistant Secretary of Defense (Networks and Information Integration)/Chief Information Officer (ASD(NII)/CIO), reports that it is developing a strategy for federating the many and varied architectures across the department‘s four mission areas”Warfighting, [Footnote 21] Business, [Footnote 22] DOD Intelligence, [Footnote 23] and Enterprise Information Environment. [Footnote 24] According to ASD(NII)/CIO officials, they are drafting a yet-to-be-released strategy for evolving DOD‘s Global Information Grid architecture, [Footnote 25] so that it provides a comprehensive architectural description of the entire DOD enterprise, including all mission areas and the relationships between and among all levels of the enterprise (e.g., mission areas, components, and programs). Figure 1 provides a simplified depiction of DOD‘s EA federation strategy. Figure 1: Simplified Depiction of the DOD Enterprise Architecture Federation Strategy: This figure is an organizational chart of the DOD Enterprise Architecture Federation Strategy. [See PDF for image] Source: GAO analysis of DOD data. [End of figure] ASD(NII)/CIO officials stated that the goal of this strategy is to improve the ability of DOD‘s mission areas, components, and programs to share architectural information. In this regard, officials stated that the DOD EA federation strategy will define (1) federation and integration concepts, (2) alignment (i.e., linking and mapping) processes, and (3) shared services. The BMA federation strategy, according to these officials, is the first mission area federation strategy, and it is their expectation that the other mission areas will develop their own respective federation strategies. DOD Roles and Responsibilities for BEA Development and Use: In 2005, the department reassigned responsibility for directing, overseeing, and executing its business transformation and systems modernization efforts to the Defense Business Systems Management Committee (DBSMC) and the Business Transformation Agency (BTA). At that time, it also adopted a tiered accountability approach to business transformation. [Footnote 26] Under tiered accountability, responsibility and accountability for business architectures and systems investment management was allocated among the DOD enterprise, [Footnote 27] component, and program levels, depending on such factors as the scope, size, and complexity of each investment. The DBSMC is chaired by the Deputy Secretary of Defense and serves as the highest-ranking governance body for business systems modernization activities. According to its charter, the DBSMC provides strategic direction and plans for the BMA in coordination with the Warfighting and Enterprise Information Environment Mission Areas. The DBSMC is also responsible for reviewing and approving the BEA and the ETP. In addition, the DBSMC recommends policies and procedures required to integrate DOD business transformation and attain cross-department, end- to-end interoperability of business systems and processes. The BTA operates under the authority, direction, and control of the DBSMC and reports to the Under Secretary of Defense for Acquisition, Technology, and Logistics in the incumbent‘s capacity as the vice chair of the DBSMC. Oversight for this agency is provided by the Deputy Under Secretary of Defense for Business Transformation, and day-to-day management is provided by the director. The BTA‘s primary responsibility is to lead and coordinate business transformation efforts across the department. Regarding the BEA, the BTA is responsible for (1) maintaining and updating the department‘s architecture; (2) ensuring that functional priorities and requirements of various defense components, such as the Department of the Army and Defense Logistics Agency (DLA), are reflected in the architecture; and (3) ensuring the adoption of DOD-wide information and process standards as defined in the architecture. Under DOD‘s tiered accountability approach to systems modernization, components are responsible for defining their respective component architectures and transition plans while complying with BEA and ETP policy and requirements. Similarly, program managers are responsible for developing program-level architectures and transition plans and ensuring integration with the architectures and transition plans developed and executed at the DOD enterprise and component levels. Summary of GAO‘s Prior Reviews on DOD‘s Architecture Efforts: Between May 2001 and July 2005, we reported on DOD‘s efforts to develop an architecture and identified serious problems and concerns with the department‘s architecture program, including the lack of specific plans outlining how DOD plans to extend and evolve the architecture to include the missing scope and detail. [Footnote 28] To address these concerns, in September 2003 [Footnote 29] we recommended that DOD develop a well-defined near-term plan for extending and evolving the architecture and ensure that this plan includes addressing our recommendations, defining roles and responsibilities of all stakeholders involved in extending and evolving the architecture, explaining dependencies among planned activities, and defining measures of progress for the activities. In response to our recommendations, in 2005, DOD adopted a 6-month incremental approach to developing its enterprise architecture and released version 3.0 of the BEA and the ETP in September 2005, describing them as the initial baselines. DOD further released version 3.1 on March 15, 2006, and version 4.0 on September 28, 2006. As we have previously reported, [Footnote 30] these incremental versions have provided additional content and clarity and resolved limitations that we identified in the prior versions. For example, DOD reports that version 4.0 begins to define a key business process area missing from prior versions”the planning, programming, and budgeting process area. In this regard, according to DOD, the architecture includes departmental and other federal planning, programming, and budgeting guidance (e.g., OMB Circular A-11) and some high-level activities associated with this area. In addition, DOD reports that version 4.0 included restructured business process models to reduce data redundancy and ensure adherence to process modeling standards (e.g., eliminated numerous process modeling standards violations and stand-alone process steps with no linkages). We concluded, however, that these incremental versions were still not sufficiently complete to effectively and efficiently guide and constrain business system investments across the department. In particular, we reported that the BEA was not yet adequately linked to the component architectures and transition plans, which is important given that the department (1) had previously announced that it had adopted a federated approach to developing and implementing the architecture and (2) had yet to address our recommendation from September 2003 [Footnote 31] for developing an architecture development management plan that defined how it intended to extend and evolve its BEA. Accordingly, in May 2006 [Footnote 32] we recommended that DOD submit an enterprise architecture development management plan to defense congressional committees. We stated that at a minimum, the plan should define what the department‘s incremental improvements to the architecture and transition plan would be and how and when they would be accomplished, including what (and when) architecture and transition plan scope and content and architecture compliance criteria would be added into which versions. In addition, we stated that the plan should include an explicit purpose and scope for each version of the architecture, along with milestones, resource needs, and performance measures for each planned version, with particular focus and clarity on the near-term versions. In response, DOD stated that, in the future, the ETP and annual report to Congress would provide additional high- level milestones for BTA activities, including the additional detail for the capability improvements to be addressed by the BEA. Our August 2006 [Footnote 33] report on the maturity of federal agency enterprise architecture programs, including those of the military departments, reemphasized the importance of DOD having an effective plan for federating its BEA. Specifically, the August report showed that the Departments of the Air Force, Army, and Navy had not satisfied about 30, 55, and 30 percent, respectively, of the 31 core elements in our Enterprise Architecture Management Maturity Framework, which is a five-stage model for effectively managing architecture governance, content, use, and measurement. [Footnote 34] In addition, the Army had only fully satisfied 1 of the 31 core elements. [Footnote 35] (See table 1 for the number of elements that were fully, partially, and not satisfied by each of the military departments.) Table 1: Number of Framework Elements That Were Fully, Partially, and Not Satisfied by the Military Departments: Military departments: Air Force; Fully satisfied: 14; Partially satisfied: 8; Not satisfied: 9. Military departments: Army; Fully satisfied: 1; Partially satisfied: 13; Not satisfied: 17. Military departments: Navy; Fully satisfied: 10; Partially satisfied: 12; Not satisfied: 9. Military departments: Total; Fully satisfied: 25; Partially satisfied: 33; Not satisfied: 35. Source: GAO. [End of table] By comparison, the other major federal departments and agencies that we reviewed had as a whole fully satisfied about 67 percent of the framework‘s core elements. Among the key elements that all three military departments had not fully satisfied were developing architecture products that describe their respective target architectural environments and developing transition plans for migrating to a target environment. Furthermore, while the military departments had partially satisfied between 8 and 13 core elements in our framework, we reported that partially satisfied elements are not necessarily easy to satisfy fully, such as those that address architecture content and thus have important implications for the quality and usability of an architecture. To assist the military departments in addressing enterprise architecture challenges and managing their architecture programs, we recommended that the military departments develop and implement plans for fully satisfying each of the conditions in our framework. The department generally agreed with our findings and recommendations. DOD Is in the Early Stages of Federating Its BEA and Much Remains to Be Decided and Accomplished: DOD‘s BMA federation strategy provides a foundation on which to build and align DOD‘s parent business architecture (the BEA) with its subordinate architectures (i.e., component- and program-level architectures). In particular, this strategy (1) states the department‘s federated architecture goals; (2) describes federation concepts that are to be applied; and (3) includes high-level activities, capabilities, products, and services that are intended to facilitate implementation of the concepts. However, DOD has yet to define the details needed to execute the strategy, such as how the architecture federation will be governed; how alignment with the DOD EA federation strategy and other potential mission area federation strategies will be achieved; how component architectures‘ alignment with incremental versions of the BEA will be achieved; how shared services will be identified, exposed, and subscribed to; and what milestones will be used to measure progress and results. According to BTA program officials, including the chief technical officer, the department is in the early stages of defining and implementing its strategy and intends to develop more detailed plans. As a result, much remains to be decided and accomplished before DOD will have in place the means to create a federated architecture and thus be able to satisfy both our prior recommendations and legislative requirements aimed at adopting an architecture-centric approach to departmentwide business systems investment management. BMA Federation Strategy Describes Goals, Concepts, and High-level Activities: BTA released the BMA federation strategy in September 2006. According to the strategy, its purpose is to expand on the DOD EA federation strategy and provide details on how various aspects of the federation will be applied within the department‘s BMA. In this regard, the BMA strategy cites the following four goals: * establish a capability to search for data in member architectures that may be relevant for analysis, reference, or reuse; * develop a consistent set of standards for architecture configuration management that will enable users to determine the development status and quality of data in various architectures; * establish a standard methodology for specifying linkages among existing component architectures that were developed using different tools and that are maintained in independent repositories; and: * develop a standard methodology to reuse capabilities described by various architectures. To assist in accomplishing these goals, the strategy describes three concepts that are to be applied. 1. Tiered accountability, which provides for architecture development at each of the department‘s organizational levels. Under this concept, each level or tier”enterprise, component, and program”has its own unique goals as well as responsibilities to the tiers above and below it. More specifically, the BTA has responsibility for the enterprise tier, including common, DOD-wide requirements and standards, while components and programs are responsible for defining component- and program-level architecture requirements and standards for their respective tiers of responsibility that are aligned with the departmentwide requirements and standards. As such, this concept introduces the need for autonomy, while also seeking to ensure linkages and alignment from the program level through the component level to the enterprise level. 2. Net-centricity, which provides for seamless and timely accessibility to information where and when needed via the department‘s interconnected network environment. This concept includes infrastructure, systems, processes, and people and is intended to ensure that users (i.e., people, applications, and platforms) of information at any level can both take what they need and contribute what they know. 3. Federating DOD architectures, which provides for linking or aligning different architectures via the mapping of common architectural information. This concept advocates subordinate architecture alignment to the parent architecture(s). Figure 2 shows a simplified version of DOD‘s BMA federated architecture. Figure 2: Simplified Diagram of DOD‘s Business Mission Area Federated Architecture: [See PDF for image] Source: GAO analysis of DOD data. [End of figure] To support the achievement of its goals and implementation of its concepts, the strategy also describes three categories of high-level activities, capabilities, products, and services”governance, [Footnote 36] federating architecture operational views, [Footnote 37] and federating architecture systems views. [Footnote 38] Table 2 shows the strategy‘s operational and systems view related activities, capabilities, products, and services. Table 2: High-level Steps for Achieving Operational and Systems View Federation: Steps for achieving operational view federation: * Support and participate in DOD‘s pilot efforts to demonstrate capability to search and navigate across the various DOD architectures. * Implement the framework through a pilot with the DLA to demonstrate how the enterprise priorities are being addressed comprehensively within the defense agencies. Map components‘ core systems to the BEA. * Implement the architecture traceability tool and continue to add functionality according to user requirements to support BEA compliance. Release the traceability tool as a Web tool accessible through the BTA portal. Steps for achieving systems view federation: * Incrementally build the common foundation infrastructure for a SOA environment by leveraging the core enterprise services, such as information assurance. Define standards for building the technical infrastructure. Stand up an initial set of infrastructure services to support ’leave-in-place“ pilots. Acquire, develop, or deploy a Business Mission Area portal. * Schedule and implement a series of ’leave-in-place“ federation pilots that can offer services and capabilities. Leverage and use the Enterprise Information Environment Mission Area core enterprise services. Bring the services that are developed as part of the pilots into the technical infrastructure, as appropriate. * Establish a governance infrastructure to establish and monitor the policies, standards, and procedures necessary for the operation of the technical infrastructure and environment. Leverage existing architecture governance structures or include a new review board to focus on systems federation requirements. * Develop an education program for stakeholders across DOD. Develop and deliver curriculum to each target stakeholder over the next year and address areas such as SOA, net-centricity, and overall federation strategy. Source: DOD. [End of table] The BMA Federation Strategy Is Missing Executable Details: Relevant architecture management guidance [Footnote 39] states that organizations should develop executable architecture development management plans and that these plans should specify, among other things, tasks to be performed, resources needed to perform these tasks (e.g., funding, staffing, tools, and training), roles and responsibilities, time frames for completing tasks, and performance measures. As previously stated, we have recommended that DOD develop such an architecture development plan to govern the evolution and extension of the BEA. [Footnote 40] We also have previously reported that a SOA approach needs to ensure that shared systems and applications (i.e., services) are, among other things, defined, developed, exposed, and subscribed to. [Footnote 41] The high-level construct of DOD‘s BMA federation strategy and the yet- to-be-issued DOD EA federation strategy reinforces the need to implement our recommendation. In particular, the strategy defines the department‘s federated architecture goals; describes federation concepts that are to be applied; and explains high-level activities, capabilities, products, and services intended to facilitate implementation of the concepts. However, it does not adequately define the tasks needed to achieve the strategy‘s goals, including those associated with executing high-level activities and providing related capabilities, products, and services. Specifically, the strategy does not adequately address how strategy execution will be governed, including assignment of roles and responsibilities, measurement of progress and results, and provision of resources. In addition, while the BMA strategy refers to several activities that are to be provided by the yet-to-be-issued DOD EA federation strategy, it does not clearly describe the relationships, dependencies, and touch points between the two strategies. Also, the strategy does not address, among other things, how the architectures of the military departments will align with the latest version of the BEA and how DOD will identify and provide for sharing of common applications and systems across the department. Moreover, the strategy does not include milestones for executing the activities and related capabilities, products, and services. Governance Structure Is Not Clearly Defined: According to ASD(NII)/CIO officials, each mission area will be responsible for establishing its own governance structures, to include defined roles and responsibilities of its members (i.e., components and programs), and such governance disciplines as measurement of progress and results and provision of resources. Moreover, officials from DOD components, such as the DLA and the Defense Information Systems Agency (DISA), told us that clearly defined and understood federation roles and responsibilities are critical to successfully executing the BMA strategy. However, the BMA strategy does not clearly define the respective roles and responsibilities of each member of the federation (i.e., enterprise, component, and program). It also does not identify the resource commitments (e.g., funding, staffing, tools, and training) needed to execute the strategy‘s activities and deliver capabilities, products, and services, or identify how fundamental governance disciplines will be performed, including performance and progress measurement. For example: * The strategy states that the DBSMC, which is currently responsible for the approval and maintenance of the BEA, will receive updates on how component (e.g., the military departments) architectures are aligning to the BEA. However, it does not describe which organizational entities are to be responsible for providing these updates or for aligning component and program architectures to the BEA. * The strategy states that in conjunction with the DOD investment review boards, [Footnote 42] the DBSMC will set the business priorities at the enterprise level through the identification of gaps in business capabilities. By establishing these priorities, the DBSMC is to determine where and when specific capabilities are addressed within the different architectures (i.e., from BEA to program-level architectures) and is to approve recommended solutions to business capability needs. However, the strategy does not provide information on who is responsible for ensuring that component priorities fit with the overall enterprise priorities, or how the DBSMC will otherwise be provided the information it needs to fulfill its stated decision-making role. * The strategy states that BMA stakeholders will need to be trained to understand the concepts presented in the strategy and begins to identify topics, such as SOA and the overall federation strategy. However, the strategy does not identify time frames and the entity responsible for providing and overseeing such training. In addition, the strategy does not address how it will be funded and staffed. * The strategy identifies categories of high-level activities, capabilities, products, and services intended to facilitate implementation of the concepts, but it does not provide for metrics that can be used to gauge the progress and ensure that expected results are realized. Relationship to DOD EA Federation Strategy Effort Is Unclear: According to the BMA federation strategy, the DOD EA federation strategy outlines an approach for linking the repositories of all of the department‘s various architectures and enabling search and navigation across them. In addition, it states that the DOD EA federation strategy outlines a series of pilot efforts that will demonstrate this approach. However, the BMA federation strategy does not clearly define how its various activities will integrate with the activities and concepts described in the yet-to-be-issued DOD EA federation strategy, or other potential mission area federation strategies, nor does it discuss how these activities will be carried out or who will be responsible for accomplishing them. For example: * ASD(NII)/CIO officials told us that the DOD EA federation strategy will establish new responsibilities for components and programs for making architecture information understandable and accessible across the department. However, these responsibilities are not explicitly discussed in the BMA federation strategy. Therefore, it is unclear how these new responsibilities are relevant to federating the BEA. Moreover, it is unclear how the BMA roles and responsibilities relate to the yet-to-be-released EA federation strategy roles and responsibilities. * The BMA federation strategy does not define how linkages among the BEA and the various component and program architectures will be established, including whether program architectures will be linked to component architectures as well as the BEA, or if program architectures will be linked to the BEA, as is currently the case. Moreover, it is not clear if establishing these linkages will be the responsibility of the programs, components, the BTA, or ASD(NII)/CIO. Operational View Federation Does Not Address Component Architecture Alignment: According to the BMA federation strategy, it builds on the DOD EA federation strategy by proposing new tools and procedures to both identify overlaps and gaps in capabilities and ensure the compliance of all component and program architectures with the BEA. In this regard, it describes the following two tools: the Investment Management Framework, which is a spreadsheet that aligns program architectures‘ capabilities (and activities) with the BEA, and the Architecture Compliance and Requirements Traceability tool, which is an automated tool that provides programs with an interface to the BEA so that they can assess their alignment with the BEA‘s operational view content (e.g., business capabilities, activities, processes, rules, and standards). However, the strategy does not address how alignment of component architectures with the BEA is to be achieved, including what, if any, component architecture alignment guidance, criteria, and tools are to be developed and who will develop them. Specifically, while the strategy states that it provides for demonstration of operational view linkages (e.g., activities, process, and capabilities) between the BEA and both component and program architectures, the tools cited do not provide the capability to either align program architectures to component architectures or to align component architectures to the BEA. According to officials from the Air Force, Navy, and DLA, they are using the traceability tool to assess compliance of their programs with the BEA. However, this tool does not allow them to assess their programs‘ compliance with their component architectures. In contrast, Army and U.S. Transportation Command officials told us that they do not require the use of the traceability tool to assess compliance of their programs to the BEA or their component architectures. According to BTA officials, they are currently working with the Air Force and Navy to expand this tool to include component architecture alignment capabilities. Systems View Federation Lacks Key Execution Details: According to the BMA strategy, the systems view federation is the application of principles, standards, services, and infrastructure to create interoperable and reusable applications and systems. The strategy states that this will be accomplished through the delivery of services within a SOA construct, including an IT infrastructure [Footnote 43] that will expose reusable functionality to federation members and enable interoperation and interconnection of the business systems and applications that provide this functionality. The strategy notes that this operating environment will be comprised of applications, systems, metadata, [Footnote 44] and a unifying portal. According to the strategy, this environment will build on existing Enterprise Information Environment Mission Area capabilities and provide the standards, policies, and technology needed to permit BMA services to be shared with the other DOD mission areas. However, the strategy does not describe how this will be accomplished, including respective roles and responsibilities of those involved, the range of services to be shared and developed, and the standards to be used. Moreover, component officials told us that the details behind the strategy‘s SOA concepts need to be defined before a systems view federation can be achieved. More specifically: * The strategy does not clearly describe how interoperable services will be defined, developed, exposed, and subscribed to. For example, it does not delineate the specific roles and responsibilities of the military departments and defense agencies relative to defining, providing, and employing shared systems and applications. As a result, the military departments and defense agencies may pursue duplicative efforts. This is of particular concern due to the various service orientation activities already under way in the military departments and defense agencies. For example, the Air Force has chartered a Transparency Integrated Product Team to guide their SOA initiatives, and the Navy has established a Transformation Group to support its service orientation activities. This is important because a key aspect of the BMA federation strategy is reusing and leveraging both enterprise-level and component-level systems and applications. * The strategy does not relate system federation activities and capabilities to its existing ETP. In particular, while the strategy describes a number of ’leave-in-place“ pilots (systems and applications) that will be implemented during the next year to demonstrate the use of shared services, it does not describe how these relate to programs in the ETP. This is important because the chief technical officer told us that many of the enterprise-level programs being managed by the BTA and included in the ETP are to evolve into shared services. * The strategy does not describe how interface standards will be established and used for obtaining and delivering shared services. Defining and enforcing such standards are important aspects of having services that are interoperable and reusable. According to the BTA chief technical officer, these standards will need to align with the yet-to-be-issued Enterprise Information Environment Mission Area standards. Officials from the Air Force and DISA agreed that more needs to be done to define the infrastructure standards that will enable user subscription to reusable systems and applications, particularly since the military departments and DOD are moving ahead with their own SOA initiatives. Strategy Does Not Include Milestones: The strategy outlines what it refers to as a high-level road map by listing activities, capabilities, products, and services that are to be produced. (See table 2 for this high-level road map.) However, the strategy does not specify the milestones or provide specific completion dates for the activities and related capabilities, products, and services listed in its high-level road map. Instead, the strategy states that the road map began in October 2006 and that milestones will occur at approximately 3-month increments, without identifying, for example, which steps have begun and what is to be accomplished over 3 months for each of the steps. Conclusions: DOD is in the early, formative stage of federating its BEA, with much remaining to be decided and accomplished before it achieves its goals. While the goals, concepts, and related activities; capabilities; products; and services discussed in the strategy have merit and hold promise, the strategy lacks sufficient specificity for it to be executed and, therefore, must be viewed as a beginning. To the department‘s credit, it recognizes the need for greater detail surrounding how it will extend (federate) its BEA. One key to making this happen is for the department to implement our prior recommendation for having a BEA development management plan. However, the department has yet to address this recommendation. Until it does, the likelihood of effectively extending the BEA to include the military departments and defense agencies is greatly reduced. Recommendation for Executive Action: To further assist the department in evolving its BEA, we are reiterating our prior recommendation for a BEA development management plan, and augmenting it by recommending that the Secretary of Defense direct the Deputy Secretary of Defense, as the chair of the DBSMC, to task the appropriate DOD organizations, to ensure that this plan describes, at a minimum, how the BMA architecture federation will be governed; how the BMA federation strategy alignment with the DOD EA federation strategy will be achieved; how component business architectures‘ alignment with incremental versions of the BEA will be achieved; how shared services will be identified, exposed, and subscribed to; and what milestones will be used to measure progress and results. Agency Comments and Our Evaluation: In written comments on a draft of this report, signed by the DOD Deputy Chief Information Officer and the Deputy Under Secretary of Defense (Business Transformation) and reprinted in appendix II, the department stated that it largely disagrees with our recommendation and added that while the BMA played a leading role in defining the department‘s approach to architecture federation and a service-oriented architecture, the impact of the issues discussed in this report goes beyond the scope of the business systems modernization. DOD also stated that any analysis of architecture federation should begin with the department‘s approach and not the BMA, since the BMA federation strategy was written as an addendum to an enterprise approach. However, DOD added that it recognizes that our analysis was complicated by the fact that many of the enterprise-level strategy and governance documents, to which the BMA must comply, have yet to be issued. The department also made the following specific comments on the five elements in our recommendation. First, DOD stated that it partially concurs with the element relating to architecture federation. According to DOD, responsibility for developing the policy and guidance regarding how architectures are to be managed within its federated environment lies with the ASD(NII)/CIO; officials acknowledge the current lack of such guidance and stated that this will be addressed with the issuance of the DOD EA federation strategy. As such, the department recommends that we address our recommendation to ASD(NII)/CIO. We agree on the current lack of and the need to develop policies and guidance describing how the federation will be governed; however, our recommendation is not intended to dictate who should develop the policies or guidance for managing architectures within a federated environment. Rather, it is focused on developing plans that describe how the BMA will adopt and implement the policies and guidance relating to federation governance. Second, the department stated that it nonconcurs with the element relating to ensuring alignment with other federation strategies. According to DOD, there is a single architecture federation strategy for the department”the DOD EA federation strategy”and other architecture federation strategies supplement this overarching strategy. As such, it stated that this element of our recommendation is not needed. We disagree. While we do not question the department‘s comment about the relationships among the strategies, we believe that this element of our recommendation is needed because its intent is to recognize these relationships by promoting collaboration and ensuring linkages among the various strategies. Third, DOD stated that it nonconcurs with the element relating to component architecture alignment with incremental versions of the BEA. According to DOD, this element has been implemented both in policy and execution to comply with legislative requirements, to include DOD‘s development and use of the Architecture Compliance and Requirements Traceability tool. It also added that the Departments of the Air Force, Army, and Navy have mandated the use of this tool to assess compliance of their systems and architectures with the BEA. We disagree. The National Defense Authorization Act for Fiscal Year 2005 includes a requirement for ensuring that all business systems in excess of $1 million be certified as being in compliance with the BEA; the architecture traceability tool provides a mechanism for asserting only system compliance and not component architecture compliance. In addition, according to officials from the Air Force and Army, while they are encouraging the use of the tool for assessing compliance of their systems with the BEA, they have not mandated its use and are not using it to assess compliance of their architectures with the BEA. Moreover, officials from the Air Force further stated that they have not mandated the use of this tool because it does not provide the capability to map the Air Force architecture with the BEA. While we recognize DOD‘s efforts to align programs to the BEA, our recommendation focuses on the lack of a discussion in the BMA federation strategy on how component architectures (military departments and defense agencies) will be linked to the BEA, including the lack of component architecture alignment guidance, criteria, and tools. Fourth, the department stated that it partially concurs with the element relating to the identification and management of shared services. According to DOD, each mission area or component is responsible for identifying its own services requirements, and the ASD(NII)/CIO is responsible for defining the overall approach to how these services will be managed. As such, the department recommends that our recommendation be directed to the ASD(NII)/CIO. We agree on the need for guidance describing how shared services will be identified and managed; however, our recommendation is not intended to dictate who should develop the policies or guidance for managing shared services within a federated environment. Rather, it is focused on developing plans that describe how the BMA will adopt and implement the policies and guidance relating to service orientation. As stated in the report, this is important because a key aspect of the BMA federation strategy is to reuse and leverage both enterprise-level and component-level systems and applications. Fifth, DOD stated that it nonconcurs with the element relating to milestones. According to DOD, milestones for gauging progress are and will continue to be monitored in the department‘s enterprise transition plan. As such, it stated that it is unclear how the need to describe what milestones will be used relates to the topics in the report. While we have previously recognized that the transition plan provides information on progress on major investments over the last 6 months”including key accomplishments and milestones attained, this element of our recommendation is intended to address the lack of measures (e.g., return on investment of service-oriented architecture service reuse) or specific completion dates for the activities and related capabilities, products, and services that are to be produced for federating the Business Mission Area. To further ensure that our recommendation is properly interpreted and implemented, and to address DOD‘s comments about directing the recommendation to the appropriate parties, we have slightly modified our recommendation. We are sending copies of this report to interested congressional committees; the Director, Office of Management and Budget; the Secretary of Defense; the Deputy Secretary of Defense; the Under Secretary of Defense for Acquisition, Technology, and Logistics; the Under Secretary of Defense (Comptroller); the Assistant Secretary of Defense (Networks and Information Integration)/Chief Information Officer; the Under Secretary of Defense (Personnel and Readiness); and the Director, Defense Finance and Accounting Service. We will also make copies available to others on request. In addition, this report will also be available at no charge on our Web site at [hyperlink, http://www.gao.gov]. If you have any questions concerning this information, please contact me at (202) 512-3439 or by e-mail at hiter@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III. Signed by: Randolph C. Hite: Director: Information Technology Architecture and Systems Issues: List of Committees: The Honorable Carl Levin: Chairman: The Honorable John McCain: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Daniel Inouye: Chairman: The Honorable Ted Stevens: Ranking Member: Committee on Appropriations: United States Senate: The Honorable Ike Skelton: Chairman: The Honorable Duncan Hunter: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable John P. Murtha: Chairman: The Honorable C.W. Bill Young: Ranking Member: Committee on Appropriations: House of Representatives: [End of section] Appendix I: Objective, Scope, and Methodology: Our objective was to determine what progress the Department of Defense (DOD) has made in defining its Business Mission Area federation strategy. To accomplish our objective, we reviewed DOD‘s Business Mission Area Federation Strategy and Road Map released in September 2006, comparing the strategy and any associated implementation plans with prior findings and recommendations relative to the content of the strategy. In particular, we compared the strategy with our prior recommendations for developing an architecture development management plan to define how the department intends to extend and evolve its business enterprise architecture. In addition, we compared the strategy with our prior findings and the need to ensure that shared systems and applications (i.e., services) are, among other things, defined, developed, exposed, and subscribed to via well-defined and standardized interfaces. Furthermore, we reviewed available information on activities, capabilities, products, and services associated with the federation strategy, such as the Investment Management Framework and the Architecture Compliance and Requirements Traceability User‘s Guide. In addition, we interviewed key program officials, including the director of the Business Transformation Agency‘s Investment Management Directorate and the chief technical officer and representatives from the Office of the Assistant Secretary of Defense (Networks and Information Integration)/Chief Information Officer, and the Departments of the Air Force, Army, and Navy; the Defense Logistics Agency and Defense Information Systems Agency; and the United States Transportation Command, to obtain an understanding of the steps taken and required to develop and execute the federation strategy. We conducted our work at DOD headquarters offices in Arlington, Virginia, from August 2006 through March 2007 in accordance with generally accepted government auditing standards. [End of section] Appendix II: Comments from the Department of Defense: Office Of The Under Secretary Of Defense: 3000 Defense Pentagon: Acquisition, Technology And Logistics: Washington, DC 20301-3000: April 3, 2007: Mr. Randolph Hite: Director: Information Technology Architecture and Systems Issues: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Hite: This is the Department of Defense (DoD) response to the GAO draft report 07-451, "Business Systems Modernization: Strategy for Evolving DOD's Business Enterprise Architecture Offers Conceptual Approach, But Execution Details Needed," dated March 1, 2007, (GAO Code 310628). This report examines two different, but related topics covered in the Business Mission Area (BMA) Federation Strategy, published last summer - the Department's approach to Architecture Federation and the evolution of the Services-Oriented Architecture (SOA) approach to creating an interoperable information environment across DoD. While both of these concepts have gained considerable momentum and have been evolving steadily within the various Mission Areas and Components of the Department, it is only recently that sufficient consensus around them has been established that specific strategies related to their global use and governance were deemed either appropriate or indeed possible. While the Business Mission Area has played a leading role in defining the Department's position on both Architecture Federation and SOA , the impact of these issues spans far beyond the scope of Business Systems Modernization. Any examination of these concepts must begin with the Department's approach, not the BMA's. The BMA Federation Strategy was written, not as stand alone documentation of the two concepts, but as an addendum to an Enterprise approach, detailing only those aspects specific to BMA requirements that would not be included in the DoD's enterprise approach. This being said, the Department recognizes that GAO's challenge in this study is complicated by the fact that many of the Enterprise-level strategy and governance documents, i.e. those to which BMA must comply, are still in draft mode. The Department did, however, share draft documents where possible to provide the broadest overview. A brief overview of the Department's position on these two topics follows: Architecture Federation – The Department's approach to Architecture Federation is described in the DoD Enterprise Architecture Federation Strategy. This document, although currently in draft (provided to GAO), has been widely circulated across the Department. It describes both the mechanisms that enable federation to exist among various Mission Area, Component and Program architectures and the responsibilities of each "tier" in ensuring that their architectures are compliant with enterprise standards. The final publication of this document is a near term priority of the DoD CIO. Other architecture federation strategies within the Department exist only to provide additional information, specific to an individual Mission Area or Component– not to repeat enterprise guidance. For example, the only discussion in the BMA Federation Strategy regarding architecture federation is the introduction of ACART, a tool used to assess compliance of Component and program business architectures with the BEA, architecture compliance being a key element to meeting FY05 NDAA requirements All three Military Departments have mandated and are currently using ACART within their investment management processes to assess and ensure compliance of their systems and architectures with the BEA. In all other respects the BMA must align with the DoD Enterprise Architecture Federation Strategy. For reference, through the BEA, ETP and other guidance, the BMA has met or exceeded all requirements placed on mission area architectures in the draft DoD Enterprise Architecture Federation Strategy. Service-Oriented Architectures – This approach to information management and application development has recently gained great momentum both within the Department and the private sector. GAO's questions and recommendations regarding SOA generally fall into two categories: 1) how service opportunities / needs are identified, and 2) how services, once developed, are managed within the Department's infrastructure. The responsibility for these two activities lies in very different areas. * Service Identification - Each Mission area or Component identifies its own services requirement based on its internal analytical and investment management process. For the BMA, this process is described in the Business Transformation Guidance (BTG). While SOA services are not yet called out specifically in the BTG, the process for identifying the need and opportunity for IT services does not differ significantly from that of identifying other business capability information technology needs. Updates on the progress of identifying services, including specific milestones will be included in future editions of the Enterprise Transition Plan (ETP) along with all other transformation and system improvement milestones. * Services Management – The DoD CIO is responsible for defining the overall approach to how SOA services will be managed within the Department's infrastructure. Department-wide guidance and policies on using, operating, and managing services are currently in development. The DoD CIO is partnering with the DNI CIO, based on input from DoD Components, Mission Areas, and the Intelligence Community, to ensure consistent guidance is provided within relevant security domains, appropriate processes are implemented, and common governance forums are used to the maximum extent practible. Attached are the Department's specific responses to GAO recommendations regarding Architecture Federation and Services-Oriented Architecture. Where the Department "non concurs", we believe that the recommendation has either already been met in its entirety through both policy and implementation or duplicates or is embedded in other prior GAO recommendations. Where the Department "partially concurs", our concerns are generally not with the core intent of the recommendation, but rather that is it worded such that it: 1. Is clearly achievable (i.e. worded in a fashion that allows DoD to take specific, reasonably-scoped steps in order to address and close the recommendation; 2. Is directed to the appropriate parties within the Department, specifically those who actually have authority over and the ability to affect solutions to the issues raised in the recommendation. In several cases, GAO's recommendations ask for DoD Enterprise guidance or policy on issues, in which case there is no Business Mission Area-specific component to the solution. GAO continues to be a valuable and constructive partner in the Department's efforts to transform internal business operations. Analysis of our plans and activities, as well as recommendations for refinement, provides important learning on best practices as we move forward. We especially appreciate the support and recognition for the Department's continued progress in driving success. We welcome GAO's insights and look forward to their participation in our future efforts. Signed by: David Wennergren: DoD Deputy Chief Information Officer: Signed by: Paul A. Brinkley: Deputy Under Secretary of Defense (Business Transformation): Gao Draft Report Dated March 1, 2007: GAO-07-451 (GAO Code 310628): Recommendation 1: The GAO is augmenting a prior recommendation regarding the business enterprise architecture (BEA) development management plan by recommending that the Secretary of Defense direct the Deputy Secretary of Defense, to have the appropriate DoD organizations, including ASD(NII)/CIO and the Director of the Business Transformation Agency (BTA) ensure that the BEA plan describes, how architecture federation will be governed. DOD Response: Partially Concur – Policy and guidance regarding how architectures are to be managed within the Department's federated environment is the responsibility of the ASD(NII)/CIO. Such policy and guidance shapes how all Mission Areas and Components will participate. There are no requirements or responsibilities for the Business Mission Area that differ from those of other mission areas. While there is a current gap in the Enterprise-wide guidance, DoD has acknowledged the shortfall and it will soon be addressed by the formal issuance of the DoD Enterprise Architecture Federation Strategy, of which the GAO has the current draft. The BMA's Federation Strategy's only contribution to the concept of Architecture Federation is to augment the draft DoD Enterprise Architecture Federation Strategy by introducing the tool (ACART) that will be used to facilitate assessment of compliance of Component and Program business architectures with the BEA during the IRB process, consistent with FY05 NDAA requirements. In every other respect, the BMA adheres to architecture federation rules established in the draft DoD Enterprise Architecture Federation Strategy. It is not the role of the BMA Federation Strategy to repeat the enterprise strategy or explain how the enterprise strategy is being implemented. This recommendation would be more appropriately directed if stated as follows: ".......that the DEPSECDEF direct the ASD(NII)/CIO to issue the appropriate enterprise guidance that describes how architecture federation will be managed and governed across the Department of Defense. Each of the Department's Mission Areas (including the BMA representing the BEA) and Components should then be charged with building their architecture plans in compliance with this guidance." Recommendation 2: The GAO is augmenting a prior recommendation regarding the BEA development management plan by recommending that the Secretary of Defense direct the Deputy Secretary of Defense, to have the appropriate DoD organizations, including ASD(NII)/CIO and the Director of the BTA ensure that the BEA plan describes how alignment with other federation strategies will be achieved. (p. 30/GAO Draft Report) DOD Response: Non Concur – There is a single Architecture Federation Strategy for the Department of the Defense. That strategy, DoD Enterprise Architecture Federation Strategy, is currently in draft and was provided to GAO during the audit. This strategy describes the principles guiding how architecture federation will work and details the roles and responsibilities of Mission Area, Component and program architectures within the overall environment. It cannot be overstated that federation adheres to tiered accountability principles, where each tier is responsible for aligning its own architecture and transformation into DoD's strategic direction. In that context, all other architecture federation "strategies" within the Department exist for the sole purpose of supplementing the DoD Enterprise Architecture Federation Strategy, to provide Mission Area or Component-specific information that would not be appropriate in the enterprise strategy. The BMA Federation Strategy is a good example of this in that it only addresses tools used in assessing BEA compliance in order to meet FY05 NDAA requirements. Given these facts, this recommendation is not needed, or at best is already implied/embedded in the previous recommendation. Recommendation 3: The GAO is augmenting a prior recommendation regarding the BEA development management plan by recommending that the Secretary of Defense direct the Deputy Secretary of Defense, to have the appropriate DoD organizations, including ASD(NII)/CIO and the Director of the BTA ensure that the BEA plan describes how component architectures alignment with incremental versions of the BEA will be achieved. (p. 30/GAO Draft Report) DOD Response: Non-Concur – This recommendation has already been fully implemented both in policy and execution. The FY05 NDAA requires that all business systems be certified as compliant with the enterprise-wide Business Enterprise Architecture. The Department has issued appropriate guidance to that affect, both in the form of guidance as to what is required to adhere to the IRB process as well as guidance specific to assessing architecture compliance. These guidances are reviewed and modified, as necessary, on an annual basis to ensure they are up-to- date with current learning, practice, tools and versions of the architecture. The Department has gone further and developed a specific tool (ACART), which is continually updated to reflect the most recent release of the BEA, to help the Component assess compliance of their Component or system architectures with the BEA's current release. This tool is currently in use widely across the Department and has been specifically mandated by Directive by each of the military services as the tool that must be used to assess compliance with the BEA. Recommendation 4: The GAO is augmenting a prior recommendation regarding the BEA development management plan by recommending that the Secretary of Defense direct the Deputy Secretary of Defense, to have the appropriate DoD organizations, including ASD(NII)/CIO and the Director of the BTA ensure that the BEA plan describes how shared services will be identified, exposed, and subscribed to. (p. 30/GAO Draft Report) DOD Response: Partially Concur – This recommendation actually asks two very different questions: 1) how service opportunities / needs are identified, and 2) how services, once developed, are managed within the Department's infrastructure. The responsibility for these two sets of issues lies in very different areas. * Service Identification - Each mission area or component identifies its own services requirement based on its internal analytical and investment management process. For the BMA, this process is described in the Business Transformation Guidance (BTG). While SOA services are not yet called out specifically in the BTG, the process for identifying the need and opportunity for IT services does not differ significantly from that of identifying other business capability information technology needs. Updates on the progress of identifying services, including specific milestones will be included in future editions of the Enterprise Transition Plan (ETP) along with all other transformation and system improvement milestones. * Services Management – The DoD CIO is responsible for defining the overall approach to how SOA services will be managed within the Department's infrastructure. Department-wide guidance and policies on using, operating, and managing services are currently in development. The DoD CIO is partnering with the DNI CIO, based on input from DoD Components, Mission Areas, and the Intelligence Community, to ensure consistent guidance is provided within relevant security domains, appropriate processes are implemented, and common governance forums are used to the maximum extent practible. Given these facts, the Department believes that the GAO's recommendation would be more effective if reworded to direct the ASD(NII)/CIO to issue enterprise policy or guidance that establishes the framework within which all SOA services (including those identified by the BMA) will be managed and direct all Mission Areas and Components to adhere to this direction. No further tasking is required for the BMA. Recommendation 5: The GAO is augmenting a prior recommendation regarding the BEA development management plan by recommending that the Secretary of Defense direct the Deputy Secretary of Defense, to have the appropriate DoD organizations, including ASD(NII)/CIO and the Director of the BTA ensure that the BEA plan describes what milestones will be used to measure progress and results. (p. 30/GAO Draft Report) DOD Response: Non-Concur – It is unclear as to how this recommendation specifically relates to the topics addressed in this report. Looking at this strictly from a BMA perspective: 1. Architecture Federation - The BEA currently meets those milestones that have been established for participating in DoD's federated architecture in the draft DoD Enterprise Architecture Federation Strategy. The BEA exists and is appropriately registered in the Department's architecture repository (DARS). There is a clearly defined mechanism in place that allows for assessment of compliance of other architectures within the federated environment with the BEA. Other future architecture federation requirements, such as the creation of a mission area taxonomy have also been met (in the BEA's case through the OV-5). As further architecture federation requirements are identified, these will be monitored in the BMA's Enterprise Transition Plan along with all other Business Transformation program milestones. 2. Services Development - Following the path of leading government and commercial organizations worldwide, DoD is enabling business agility through a modular, federated integration of applications – SOA. This approach allows service development and deployment to become more consistent across the enterprise. Within that context, the development of specific SOA services is little different from the development and implementation of other capabilities. New applications can be developed and deployed as services and existing applications can provide and consume these services. They result from the same process detailed in the Business Transformation Guidance (BTG) and will (and already have been) be detailed in the milestones detailed in the BMA's Enterprise Transition Plan. Repeating the milestones in the BEA would be a redundant effort and is unnecessary. Given this, it is the Department's belief that this recommendation asks the Department to do something the GAO has already recognized that the Department is doing. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Randolph C. Hite, (202) 512-3439 or hiter@gao.gov: Staff Acknowledgments: In addition to the contact person named above, key contributors to this report were Neil Doherty, Nancy Glover, Michael Holland, Neelaxi Lakhmani (Assistant Director), Anh Le, Jacqueline Mai, and Jennifer Stavros-Turner. [End of section] Footnotes: [1] Business systems are information systems that include financial and nonfinancial systems and support DOD‘s business operations, such as civilian personnel, finance, health, logistics, military personnel, procurement, and transportation. [2] GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.: January 2007). [3] An EA, or modernization blueprint, provides a clear and comprehensive picture of an entity, whether it is an organization (e.g., federal department or agency) or a functional or mission area that cuts across more than one organization (e.g., financial management). This picture consists of snapshots of the enterprise‘s current ’As Is“ operational and technological environment and its target or ’To Be“ environment, as well as a capital investment road map for transitioning from the current to the target environment. These snapshots further consist of ’views,“ which are one or more architecture products that provide conceptual or logical representations of the enterprise. [4] GAO, Information Technology: Architecture Needed to Guide Modernization of DOD‘s Financial Operations, GAO-01-525 (Washington, D.C.: May 17, 2001). [5] See, for example, GAO-01-525; GAO, DOD Business Systems Modernization: Improvements to Enterprise Architecture Development and Implementation Efforts Needed, GAO-03-458 (Washington, D.C.: Feb. 28, 2003); Information Technology: Observations on Department of Defense‘s Draft Enterprise Architecture, GAO-03-571R (Washington, D.C.: Mar. 28, 2003); Business Systems Modernization: Summary of GAO‘s Assessment of the Department of Defense‘s Initial Business Enterprise Architecture, GAO-03-877R (Washington, D.C.: July 7, 2003); DOD Business Systems Modernization: Important Progress Made to Develop Business Enterprise Architecture, but Much Work Remains, GAO-03-1018 (Washington, D.C.: Sept. 19, 2003); DOD Business Systems Modernization: Limited Progress in Development of Business Enterprise Architecture and Oversight of Information Technology Investments, GAO-04-731R (Washington, D.C.: May 17, 2004); DOD Business Systems Modernization: Billions Being Invested without Adequate Oversight, GAO-05-381 (Washington, D.C.: Apr. 29, 2005); and DOD Business Systems Modernization: Long-standing Weaknesses in Enterprise Architecture Development Need to Be Addressed, GAO-05-702 (Washington, D.C.: July 22, 2005). [6] Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct. 28, 2004) (codified in part at 10 U.S.C. § 2222). [7] GAO, Business Systems Modernization: DOD Continues to Improve Institutional Approach, but Further Steps Needed, GAO-06-658 (Washington, D.C.: May 15, 2006). [8] GAO-06-658. [9] GAO-06-658. [10] See, for example, GAO, Defense Inventory: Opportunities Exist to Improve Spare Parts Support Aboard Deployed Navy Ships, GAO-03-887 (Washington, D.C.: Aug. 29, 2003); Military Pay: Army National Guard Personnel Mobilized to Active Duty Experienced Significant Pay Problems, GAO-04-89 (Washington, D.C.: Nov. 13, 2003); and DOD Travel Cards: Control Weaknesses Resulted in Millions of Dollars of Improper Payments, GAO-04-576 (Washington, D.C.: June 9, 2004). [11] GAO-07-310. [12] These high-risk areas include DOD‘s overall approach to business transformation, business systems modernization, financial management, the personnel security clearance program, supply chain management, support infrastructure management, weapon systems acquisition, and contract management. [13] The 7 governmentwide high-risk areas are as follows: (1) disability programs, (2) ensuring the effective protection of technologies critical to U.S. national security interests, (3) interagency contracting, (4) information systems and critical infrastructure, (5) information-sharing for homeland security, (6) human capital, and (7) real property. [14] The Clinger-Cohen Act of 1996, 40 U.S.C. § 11315(b)(2). [15] The E-Government Act of 2002, Pub. L. No. 107-347 (Dec. 17, 2002). [16] GAO, Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity, GAO-04-394G (Washington, D.C.: March 2004); CIO Council, A Practical Guide to Federal Enterprise Architecture, Version 1.0 (February 2001); and OMB Capital Programming Guide, Version 1.0 (July 1997). [17] John A. Zachman, ’A Framework for Information Systems Architecture,“ IBM Systems Journal 26, No. 3 (1987). [18] DOD, Department of Defense Architecture Framework, Version 1.0, Volume 1 (August 2003) and Volume 2 (February 2004). [19] See, for example, GAO, Homeland Security: Efforts Under Way to Develop Enterprise Architecture, but Much Work Remains, GAO-04-777 (Washington, D.C.: Aug. 6, 2004); GAO-04-731R; Information Technology: Architecture Needed to Guide NASA‘s Financial Management Modernization, GAO-04-43 (Washington, D.C.: Nov. 21, 2003); GAO-03-1018; GAO-03-877R; Information Technology: DLA Should Strengthen Business Systems Modernization Architecture and Investment Activities, GAO-01-631 (Washington, D.C.: June 29, 2001); and Information Technology: INS Needs to Better Manage the Development of Its Enterprise Architecture, and GAO/AIMD-00-212 (Washington, D.C.: Aug. 1, 2000). [20] GAO, Information Technology: FBI Has Largely Staffed Key Modernization Program, but Strategic Approach to Managing Program‘s Human Capital Is Needed, GAO-07-19 (Washington, D.C.: Oct. 16, 2006). [21] The Warfighting Mission Area focuses on transforming how DOD achieves its mission objectives by addressing joint warfighting capabilities and providing life-cycle oversight to applicable DOD component and combatant command IT investments. [22] The Business Mission Area is responsible for ensuring that capabilities, resources, and materiel are reliably delivered to the warfighter. Specifically, the BMA addresses areas such as real property and human resources management. [23] The DOD Intelligence Mission Area is focused on establishing advanced capabilities to anticipate adversaries and includes IT investments within the military intelligence program and defense component programs of the National Intelligence Program. [24] The Enterprise Information Environment Mission Area enables the functions of the other mission areas (e.g., Warfighting Mission Area, Business Mission Area, and National Intelligence Mission Area) and encompasses all communications, computing, and core enterprise service systems, equipment, or software that provides a common information capability or service for enterprise use. [25] According to DOD, the Global Information Grid consists of a globally interconnected, end-to-end set of information capabilities, associated processes, and personnel for collecting, processing, storing, disseminating, and managing information on demand to warfighters, policymakers, and support personnel, and as such represents the department‘s IT architecture. [26] Under a tiered accountability approach, the BEA describes the envisioned processes, systems, and standards and components are responsible for defining a component-level architecture associated with their own tier of responsibility in alignment with the BEA‘s enterprisewide standards and requirements. [27] The DOD enterprise is comprised of the entities in the Office of the Secretary of Defense”the DBSMC, the BTA, and the Investment Review Boards. [28] See, for example, GAO-01-525, GAO-03-458, GAO-03-571R, GAO-03- 877R, GAO-03-1018, GAO-04-731R, GAO-05-381, and GAO-05-702. [29] GAO-03-1018. [30] GAO, Defense Business Transformation: A Comprehensive Plan, Integrated Efforts, and Sustained Leadership Are Needed to Assure Success, GAO-07-229T (Washington, D.C.: Nov. 16, 2006). [31] GAO-03-1018. [32] GAO-06-658. [33] GAO, Enterprise Architecture: Leadership Remains Key to Establishing and Leveraging Architectures for Organizational Transformation, GAO-06-831 (Washington, D.C.: Aug. 14, 2006). [34] GAO, Information Technology: A Framework for Assessing and Improving Enterprise Architecture Management (Version 1.1), GAO-03-584G (Washington, D.C.: April 2003). [35] We did not review the enterprise architecture programs at other DOD components, such as the Defense Information Systems Agency or the DLA. [36] According to the strategy, governance addresses how the BMA federated approach will be implemented across DOD components by describing the new responsibilities imposed on the existing business systems governance structures resulting from the federation. [37] The DOD Architecture Framework defines the operational view as a description of the tasks and activities, operational elements, and information exchanges required to accomplish DOD missions. Federating architecture operational views, according to the BMA strategy, is an approach for gaining visibility across the various business architectures by enabling linkages and alignment among these various BMA architectures‘ activities, processes, and data (e.g., DOD, component, and program). [38] The DOD Architecture Framework defines the systems view as a set of graphical and textual products that describe systems and interconnections providing for or supporting DOD functions. Federating architecture system views, according to the BMA strategy, is an approach for the delivery of shared business systems and information services within a SOA through an IT infrastructure. [39] See, for example, GAO-03-584G and A Practical Guide to Federal Enterprise Architecture. [40] GAO-03-1018 and GAO-06-658. [41] GAO-07-19. [42] The investment review boards serve as the oversight and investment decision-making bodies for those business capabilities that support activities under their designated areas of responsibility. [43] The strategy refers to the IT infrastructure as the business transformation infrastructure. [44] Metadata are data used to supplement information to main data. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Paul Anderson, Managing Director, AndersonP1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.