Defense Acquisitions

Department of Defense Actions on Program Manager Empowerment and Accountability Gao ID: GAO-08-62R November 9, 2007

In November 2005, we issued a report on the environment within which the Department of Defense (DOD) program managers perform their work. We identified areas where program managers believe they are insufficiently empowered to execute programs, and therefore, because much is beyond their control, accountability is difficult. We also compared department policies and practices to those of leading commercial companies we visited and discussed actions DOD could take to improve program manager accountability, while also providing them with timely support as they manage the development of weapon systems. We recommended that DOD take a number of actions to ensure program managers are well positioned to successfully execute acquisitions and be held accountable. The John Warner National Defense Authorization Act for Fiscal Year 2007 directed the Secretary of Defense to develop a comprehensive strategy for enhancing the role of DOD program managers in developing and carrying out defense acquisition programs and to revise guidance for major defense acquisition programs to address the qualifications, resources, responsibilities, tenure, and accountability of program managers for the program development and execution periods. In addition, GAO was directed to report on the actions taken by the Secretary of Defense to implement the requirements of the Act. To identify DOD actions to implement the Act, we met with DOD officials and reviewed documents they provided to determine what actions were planned to address and implement the congressional mandate, including DOD's required strategy. We reviewed existing DOD policies, directives, and guidance on the qualifications, resources, responsibilities, authority, tenure, and accountability of program managers. Finally, we made extensive use of our prior work in this and other related areas. We conducted our work from January to September 2007 in accordance with generally accepted government auditing standards.

As part of DOD's strategy to enhance the role of program managers in carrying out its major weapon system acquisitions, the department has established a policy that requires formal agreements among program managers, their acquisition executives, and the user community intended to set forth common program goals. In addition, as part of a new strategy for program manager empowerment and accountability, DOD plans a variety of actions to enhance development opportunities, provide more incentives, and arrange knowledge-sharing opportunities. For example, DOD intends to increase "just-in-time" training, establish a formal mentoring program, and establish program management forums. The department also acknowledges that any actions taken to improve accountability must be based on a foundation whereby program managers can launch and manage programs toward greater performance, rather than focusing on maintaining support and funding for individual programs. This foundation should include an overarching strategy and decision-making processes that prioritize programs based on a balanced match between customer needs and available department resources. Within its strategy and other reports, DOD has highlighted several initiatives that, if adopted and implemented properly, could provide such a foundation. For example, the new Concept Decision Point initiative, guidance for Time-Defined Acquisitions, and the establishment of Configuration Steering Boards are all designed to enable more informed decisions by key department leaders well ahead of a program's start, decisions that provide a closer match between each program's requirements and the department's resources. DOD acquisition leaders stated that any improvements to program managers' performance hinge on the success of these departmental initiatives.



GAO-08-62R, Defense Acquisitions: Department of Defense Actions on Program Manager Empowerment and Accountability This is the accessible text file for GAO report number GAO-08-62R entitled 'Defense Acquisitions: Department of Defense Actions on Program Manager Empowerment and Accountability' which was released on November 9, 2007. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. November 9, 2007: Congressional Committees: Subject: Defense Acquisitions: Department of Defense Actions on Program Manager Empowerment and Accountability: In November 2005, we issued a report on the environment within which the Department of Defense (DOD) program managers perform their work.[Footnote 1] We identified areas where program managers believe they are insufficiently empowered to execute programs, and therefore, because much is beyond their control, accountability is difficult. We also compared department policies and practices to those of leading commercial companies we visited and discussed actions DOD could take to improve program manager accountability, while also providing them with timely support as they manage the development of weapon systems. We recommended that DOD take a number of actions to ensure program managers are well positioned to successfully execute acquisitions and be held accountable. The John Warner National Defense Authorization Act for Fiscal Year 2007 directed the Secretary of Defense to develop a comprehensive strategy for enhancing the role of DOD program managers in developing and carrying out defense acquisition programs and to revise guidance for major defense acquisition programs to address the qualifications, resources, responsibilities, tenure, and accountability of program managers for the program development and execution periods.[Footnote 2] In addition, GAO was directed to report on the actions taken by the Secretary of Defense to implement the requirements of the Act. To identify DOD actions to implement the Act, we met with DOD officials and reviewed documents they provided to determine what actions were planned to address and implement the congressional mandate, including DOD's required strategy. We reviewed existing DOD policies, directives, and guidance on the qualifications, resources, responsibilities, authority, tenure, and accountability of program managers. Finally, we made extensive use of our prior work in this and other related areas. We conducted our work from January to September 2007 in accordance with generally accepted government auditing standards. Results in Brief: As part of DOD's strategy to enhance the role of program managers in carrying out its major weapon system acquisitions, the department has established a policy that requires formal agreements among program managers, their acquisition executives, and the user community intended to set forth common program goals. In addition, as part of a new strategy for program manager empowerment and accountability, DOD plans a variety of actions to enhance development opportunities, provide more incentives, and arrange knowledge-sharing opportunities. For example, DOD intends to increase "just-in-time" training, establish a formal mentoring program, and establish program management forums. The department also acknowledges that any actions taken to improve accountability must be based on a foundation whereby program managers can launch and manage programs toward greater performance, rather than focusing on maintaining support and funding for individual programs. This foundation should include an overarching strategy and decision- making processes that prioritize programs based on a balanced match between customer needs and available department resources. Within its strategy and other reports, DOD has highlighted several initiatives that, if adopted and implemented properly, could provide such a foundation. For example, the new Concept Decision Point initiative, guidance for Time-Defined Acquisitions, and the establishment of Configuration Steering Boards are all designed to enable more informed decisions by key department leaders well ahead of a program's start, decisions that provide a closer match between each program's requirements and the department's resources. DOD acquisition leaders stated that any improvements to program managers' performance hinge on the success of these departmental initiatives. Background: In our 2005 report, we recommended that DOD take a number of actions to ensure program managers are well positioned to successfully execute acquisitions and be held accountable. In turn, Congress mandated that DOD develop a comprehensive strategy for enhancing the role of DOD program managers in developing and carrying out defense acquisition programs. This strategy was to address, at a minimum: * enhanced training and educational opportunities for program managers; * increased emphasis on the mentoring of current and future program managers by experienced senior executives and program managers within the department; * improved career paths and career opportunities for program managers; * additional incentives for the recruitment and retention of highly qualified individuals to serve as program managers; * improved resources and support (including systems engineering expertise, cost estimating expertise, and software development expertise) for program managers; * improved means of collecting and disseminating best practices and lessons learned to enhance program management throughout the department; * common templates and tools to support improved data gathering and analysis for program management and oversight purposes; * increased accountability of program managers for the results of defense acquisition programs; and: * enhanced monetary and nonmonetary awards for successful accomplishment of program objectives by program managers. While DOD was required to report on its development of a comprehensive strategy for improving various aspects of program manager roles and responsibilities by July 14, 2007, we did not receive a copy of this report until September 25, 2007. The Act also required DOD to revise its guidance for major defense acquisition programs to address the qualifications, resources, responsibilities, tenure, and accountability of program managers for the program development and program execution period. At a minimum, the revised guidance was expected to address the: * need for a performance agreement between a program manager and the milestone decision authority for the program, setting forth expected parameters for cost, schedule, and performance, and appropriate commitments by the program manager and the milestone decision authority to ensure that such parameters are met; * authorities available to the program manager, including, to the extent appropriate, the authority to object to the addition of new program requirements that would be inconsistent with the parameters established when the program formally began development (Milestone B for major acquisition programs or Key Decision Point B in the case of a space program) and reflected in the performance agreement; and: * extent to which a program manager for such period should continue in the position without interruption until the delivery of the first production units of the program. In February 2007, DOD issued a report on the department's acquisition transformation initiatives and the goals established in order to achieve change.[Footnote 3] Within that report, DOD noted that every aspect of how the department does business was being assessed and streamlined to deliver improved capabilities to the warfighter and visibility to executive leadership. The report also noted the need for continuous and evolutionary changes across the DOD acquisition system, especially with regard to determining which assets and investments to acquire in order to meet desired capabilities. Future reports on acquisition transformation were expected to build on the outcomes of initiatives described in that report. New DOD Approach Includes Various Actions to Increase Accountability and Support of Program Managers: DOD's revised approach for program managers includes a number of ongoing and planned actions to increase accountability and provide more support to program managers by enhancing individual development, providing more incentives, and arranging knowledge-sharing opportunities. Related to findings and recommendations in our 2005 report, DOD previously initiated actions to increase program managers' accountability. In December 2005, DOD issued new policy to provide uniform guidance for managing positions and career development of the acquisition workforce, a policy that included program managers.[Footnote 4] The new policy identified career paths for program managers in terms of education, training, experience, and assignments. It also established requirements for program manager agreements, including specified tenure periods, when they are assigned to a program. In response to congressional direction, DOD is planning to establish other mechanisms for increasing program manager accountability. In December 2006, DOD formed a working group led by the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics (OUSD/AT&L) to address the requirements of the fiscal year 2007 Defense Authorization Act, including development of the program manager empowerment and accountability strategy. The group was made up of defense acquisition career managers and service acquisition executives from all the services as well as Defense Acquisition University representatives. The working group began by reviewing successful practices that already exist within the Office of the Secretary of Defense and the military services. Based on this review, the group planned to either adopt these practices on a wider scale or develop new practices based on past experience and new ideas. In conjunction with the working group, DOD issued a May 2007 policy memorandum to emphasize and amplify the 2005 program manager tenure and accountability policy.[Footnote 5] DOD believes this will ensure that program managers have sufficient tenure to achieve the outcomes expected of them and to provide additional goal-oriented mechanisms that improve both systemic and personal accountability. The new policy maintains existing tenure agreements and qualifications for program managers. However, it also provides for performance management agreements, which are new. The new agreements are designed to provide a documented basis for ensuring that a program manager's plans are consistent with those of the organization, that there is a common basis for understanding and accountability, that the specified plans are fully resourced and achievable, and that both the organization's and individual's responsibilities are effectively communicated. The policy states that agreements will be prepared for programs at the acquisition category I and II levels and signed by the program manager, acquisition executive, and the requirements or resource authorities. Further, the policy states that the agreements will be prepared after DOD makes an investment decision to pursue a new program and a program manager is assigned and should consistently reflect the approved program. The agreements are expected to establish achievable and measurable plans that are to be updated annually, or more frequently if the conditions that formed the basis for the agreement--such as requirements, funding, or execution plans--have changed. In September 2007, DOD released a new strategy expected to address department efforts related to a holistic approach to acquisition transformation. Specific to program manager empowerment and accountability, this strategy notes that the department is beginning initiatives, identifying enhancements, and developing implementation plans associated with actions aimed at improving support to program managers. These initiatives serve to address specific improvements in program manager development, knowledge sharing, and stability and support. The strategy further acknowledges that additional effort is ongoing and required between the Office of the Secretary of Defense and the military components to develop deliberate, enterprisewide acquisition position and career management in order to ensure effective and efficient implementation of the separate initiatives within the overall strategy. See table 1 for the additional actions DOD is currently planning to support program managers. Table 1: DOD Actions Planned to Support Program Managers: Mandate item: Training and education; Actions Planned: DOD, through the Defense Acquisition University, plans to increase "just-in-time" training for program managers. DOD also plans to deploy a new "core-plus" concept as part of the training curriculum for program managers. Mandate item: Mentoring; Actions Planned: DOD plans to establish a formal program-manager- focused mentoring program. This would be augmented with the opportunity for program managers to speak with former program managers and retired flag officers and senior executives. DOD's strategy states that each military service will develop a formalized program manager mentorship program within 18 months. Mandate item: Career paths & opportunities; Actions Planned: DOD is planning to emphasize professional development for civilian program managers and establish/expand executive development for acquisition professionals. For example, the DOD strategy states that the department is exploring the establishment of a specialized "professional" acquisition program management occupational series. Mandate item: Recruitment & retention incentives; Actions Planned: DOD plans to increase Senior Service School selection rates and expand executive development participation. The DOD strategy notes that these types of initiatives would be used to the extent possible to encourage top military and civilian members to compete for the most challenging assignments. Mandate item: Resources & support; Actions Planned: DOD plans to institute teams aligned with particular phases of an acquisition program to assist program managers as necessary. This would include program support and non-advocacy reviews, acquisition centers of excellence, and program start-up workshops at program initiation. Other plans include establishing program management forums within the department where program managers can share information with each other. One such forum would be through the Program Management Industrial Committee--developed with the National Defense Industrial Association--expected to allow discussion of program management issues from both government and industry perspectives. Mandate item: Best practices & lessons learned; Actions Planned: In addition to program management forums and assist teams discussed above, DOD plans to begin program manager "webcasts." DOD plans to link acquisition knowledge databases together to improve performance and maximize value. Mandate item: Common templates & tools; Actions Planned: DOD does not initially intend to expend effort to develop new or additional templates and tools; rather it will focus on increasing overall awareness of those already in place within the department and each military department. DOD's strategy notes that effort will be needed to improve the user interface to such tools, including information and best practices databases. Additional effort is also expected to be necessary to link knowledge systems, databases, and resources together. Mandate item: Increase accountability; Actions Planned: In addition to actions related to a May 2007 policy memorandum, DOD plans to encourage ACAT ID program managers to meet with the defense acquisition executives when first assigned. Mandate item: Monetary & non-monetary awards; Actions Planned: DOD plans to explore the use of monetary awards. However, the DOD strategy acknowledges that because monetary awards involve "sensitive compensation questions," additional research and a workforce analysis of the program manager field are currently underway. Source: Department of Defense, Under Secretary for Acquisition, Technology and Logistics, Report to: Congress on Program Manager Empowerment and Accountability (Washington, D.C.: September 2007). [End of table] DOD Practices for Program Manager Accountability Must Match Policy Efforts: Since 2003, while noting some positive changes to DOD acquisition policy, we have reported that the department's leadership needs to put necessary controls in place to ensure decision makers could make informed judgments.[Footnote 6] As we reported in 2005, DOD continues to start many more programs than it can afford and rarely prioritizes them for funding purposes.[Footnote 7] In addition, DOD allows programs to begin without establishing a formal business case. Further, technologies are not always mature at program start, requirements are not always fully defined, and cost and schedule estimates are often unrealistic--resulting in a mismatch between customer needs and available resources. As a result, program managers are often not well- positioned to successfully execute weapon acquisitions and are often relegated to program advocacy in order to ensure programs continue. DOD acknowledges, and we agree, that any initiative intended to operate within the department's existing organizational framework must be universally supported by OSD leadership and each service component or the initiative will not provide sufficient authority and control over resources to effectively influence weapon system investments. Therefore, DOD's strategy to improve acquisition outcomes rests not only with providing support for individual program managers, but more importantly on creating a departmentwide environment that promotes stability, accountability, and more informed decisionmaking. To that end, the new DOD strategy notes that successful acquisition program outcomes are not possible without continuous alignment of three principal functions: requirements, funding, and acquisition execution. In order to support improvements in these functional areas, DOD has or plans to begin several initiatives intended to address shortfalls in its existing approach to investment decisions, such as establishing an early decision gate to review proposed programs at the concept stage, testing portfolio management approaches in selected capability areas, planning time-defined acquisitions, creating configuration steering boards, and using capital budgeting accounts for programs in development. See table 2 for examples of initiatives related to requirements, funding, and acquisition execution. Table 2: DOD Initiatives on Requirements, Funding, and Acquisition Execution. Initiative: Concept decision; Description: Framework for strategic investment decisions. Initiative: Time-defined acquisitions; Description: Where feasible, first capability increment will be fielded in 5 to 6 years from investment decisions. Initiative: Human capital planning; Description: Comprehensive enterprise and specific component human capital plans to mitigate demographic and talent challenges, including the AT&L Human Capital Strategic Plan. Initiative: Requirements management certification training; Description: Portfolio of learning assets to train military and civilians who write and manage requirements for major defense acquisition programs. Initiative: Performance management; Description: Implementation of National Security Personnel System (NSPS) and the Executive and Senior Professional Pay and Performance System to enable outcomes-based performance evaluations and rewards. Initiative: Centers of excellence; Description: Continuous review and improvement of processes and practices to strengthen planning and execution of acquisition programs. Initiative: Risk-based source selection; Description: Identify and quantify risks, informing requirements development and cost estimation, and improve the assessment of contractor proposals. Initiative: Acquisition of services policy; Description: Ensure executive reviews and implementation of best practices. Initiative: Restructured Defense Acquisition Executive Summary (DAES); Description: Establish an analytical foundation to ensure effective program management with predictable acquisition outcomes consistent with user requirements. Source: Department of Defense, Under Secretary for Acquisition, Technology and Logistics, Report to Congress on Program Manager Empowerment and Accountability (Washington, D.C.: September 2007). [End of table] The DOD strategy emphasizes that initiatives designed to improve program manager performance can only be successful if the strategic objectives are accepted and implemented. This support, however, is not a given. Policies similar to the one issued by DOD in May 2007 have existed for some time, but according to OUSD and Air Force officials, have not always been practiced. For example, while DOD policy provides for program managers of major defense acquisition programs to serve as close to 4-year tenures as practicable, many serve for only 2 years. Such inconsistency between policy and practice could hinder DOD's efforts to improve program manager accountability. In fact, our work has shown that rather than lengthy assignment periods between key milestones as suggested by best practices, many of the programs we have reviewed had multiple program managers within the same milestone. For example, analysis for this review showed that for 39 major acquisition programs started since March 2001, the average time in development was about 37 months. The average tenure for program managers on those programs during that time was about 17.2 months--less than half of what is prescribed by DOD policy. According to the Director for OUSD Portfolio Systems Acquisition, support for these accountability initiatives must come both from within DOD and from the military departments. Further, the Director noted that getting this support will be a major challenge in efforts to improve empowerment and accountability for program managers and ensure consistency between policy and practice. For example, the Director noted that advocacy for significant and continuous leadership will be needed to implement the tenure and accountability agreements currently planned and, maybe more importantly, secure funding necessary to carry out those agreements. The department's strategy further states that "if our [DOD's] efforts to improve requirements and resource stability are not successful, we cannot expect program manager performance to improve nor can we justly hold our program managers accountable." Conclusions: DOD is planning to implement many new practices that will provide program managers more incentives, support, and stability as well as opportunities for sharing knowledge and lessons learned. DOD has also established initiatives designed to change the strategic environment at the portfolio level. We agree that these initiatives reflect sound business practices and, if fully and correctly implemented, could lead to better program manager performance and overall acquisition outcomes. However, DOD also acknowledges that if efforts to improve requirements and resource stability are not successful, there cannot be an expectation that changes will occur at the program manager level. Moreover, if history is any judge, the overall environment within which program managers perform their work is very difficult to change simply with policy initiatives. Unless all of the players involved with acquisitions--the requirements community, the comptroller, the Undersecretary for AT&L and perhaps most importantly, the military services--are unified in implementing these new policies from top to bottom, they will be for naught. The new practices DOD is planning to implement will not be as effective as they could be until DOD ensures that program managers are given acquisition programs that are executable--that is, programs that are the result of an integrated, portfolio-based approach to investments and that have a sound business case. Only then will program managers be placed in a better position to carry out their programs in a manner suited for successful outcomes. DOD's actions to implement the new strategy, to reinforce the importance of tenure agreements, and to gain agreement and acceptance from the military services can only be effective and implemented as intended with strong and consistent vision, direction, and advocacy from DOD leadership. Greater department- level support is important to ensure that DOD policy becomes practice, especially if program managers are to be truly empowered and held accountable for outcomes. Agency Comments: In commenting on a draft of this report, DOD provided technical comments, which were incorporated as appropriate. We will send copies of this correspondence to the Department of Defense and interested congressional committees. We will also make copies available to others upon request. In addition, the report will be available at no charge on GAO's Web site at [hyperlink, http://www.gao.gov]. Should you or your staff have any questions on matters discussed in this correspondence, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this correspondence. Principal contributors to this report were Brian Mullins, Assistant Director; Sigrid McGinty; Peter Zwanzig; Marie Ahearn; and Greg Campbell. Signed by: Michael J. Sullivan: Director: Acquisition and Sourcing Management: List of Committees: The Honorable Carl Levin: Chairman: The Honorable John McCain: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Daniel K. Inouye: Chairman: The Honorable Ted Stevens: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Ike Skelton: Chairman: The Honorable Duncan L. Hunter: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable John P. Murtha: Chairman: The Honorable C.W. Bill Young: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Footnotes: [1] GAO, Best Practices: Better Support of Weapon System Program Managers Needed to Improve Outcomes, GAO-06-110 (Washington, D.C.: Nov. 30, 2005). [2] John Warner National Defense Authorization Act for Fiscal Year 2007, Pub. L. No. 109-364, §853 (2006). [3] Department of Defense, Secretary of Defense, Defense Acquisition Transformation: Report to Congress (Washington, D.C.: February 2007). [4] Department of Defense Instruction No. 5000.66, Operation of the Defense Acquisition, Technology, and Logistics Workforce Education, Training, and Career Development Program (Dec. 21, 2005). The policy provides that the tenure period for program managers of major defense acquisition programs is the program milestone closest to 4 years or as tailored by the acquisition executive based on unique program requirements, in accordance with 10 UCS § 1734(b). [5] Under Secretary of Defense (AT&L) Memorandum, Program Management Tenure and Accountability, (May 25, 2007). [6] GAO, Defense Acquisitions: DOD's Revised Policy Emphasizes Best Practices, but More Controls Are Needed, GAO-04-53 (Washington, D.C; Nov. 2003). [7] GAO-06-110. GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.