Census Bureau
Important Activities for Improving Management of Key 2010 Decennial Acquisitions Remain to be Done Gao ID: GAO-06-444T March 1, 2006The Census Bureau plans to increase its use of automation to conduct the 2010 Decennial Census. Two key acquisitions are the Decennial Response Integration System (DRIS) and the Field Data Collection Automation program (FDCA). DRIS is expected to standardize and integrate data from census forms and other response modes. FDCA is expected to provide automation support for field data collection activities. Last year, Congress asked GAO to assess the status, plans, and management capabilities of both of these projects. In January 2006, GAO briefed Congressional staff on the results of that work. This testimony summarizes key findings from that briefing, including the status and management capabilities of each project.
The Census Bureau has initiated efforts to acquire DRIS and FDCA, key systems it needs to perform the 2010 Decennial Census. It awarded a contract for DRIS in October 2005; the system is currently in a design and development phase. The bureau expects to award a contract for FDCA development in March 2006. While both projects' life cycle cost estimates are currently considered procurement sensitive, together they are expected to make up a large portion of the $1.8 billion program to develop, test, and implement decennial census systems. Both acquisitions involve ambitious schedules in order to be able to deliver the needed functionality to support a planned 2008 census dress rehearsal and the eventual 2010 census activities. While both project offices have implemented initial acquisition management activities, neither has the full set of capabilities they need to effectively manage the acquisitions. Specifically, the DRIS project completed its solicitation activities and the FDCA project has completed most of its solicitation activities. However, activities in other management areas have been initiated but not completed. For example, the DRIS project office has established baseline requirements for the acquisition, but the bureau has not yet validated them or implemented a process for managing the requirements. Also, while the FDCA project office has initiated efforts to oversee the contractor's performance, such as requiring earned value management reporting (a project management tool that integrates cost, schedule, and scope of work to aid project planning and control) and hiring staff with contracting experience, it has not yet determined which performance measures it will use to track the contractor and its own internal project office performance. Until these and other basic contract management activities are fully implemented, both projects face increased risks that the systems will experience cost overruns, schedule delays, and performance shortfalls.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-444T, Census Bureau: Important Activities for Improving Management of Key 2010 Decennial Acquisitions Remain to be Done
This is the accessible text file for GAO report number GAO-06-444T
entitled 'Census Bureau: Important Activities for Improving Management
of Key 2010 Decennial Acquisitions Remain to be Done' which was
released on March 1, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony Before the Subcommittee on Federalism and the Census,
Committee on Government Reform, House of Representatives:
For Release on Delivery Expected at 10:00 a.m. EST Wednesday, March 1,
2006:
Census Bureau:
Important Activities for Improving Management of Key 2010 Decennial
Acquisitions Remain to be Done:
Statement of David A. Powner, Director, Information Technology
Management Issues:
GAO-06-444T:
GAO Highlights:
Highlights of GAO-06-444T, testimony before the Committee on Government
Reform, Subcommittee on Federalism and the Census:
Why GAO Did This Study:
The Census Bureau plans to increase its use of automation to conduct
the 2010 Decennial Census. Two key acquisitions are the Decennial
Response Integration System (DRIS) and the Field Data Collection
Automation program (FDCA). DRIS is expected to standardize and
integrate data from census forms and other response modes. FDCA is
expected to provide automation support for field data collection
activities. Last year, you asked GAO to assess the status, plans, and
management capabilities of both of these projects. In January 2006, GAO
briefed the subcommittee staff on the results of that work. At your
request, this testimony summarizes key findings from that briefing,
including the status and management capabilities of each project.
What GAO Found:
The Census Bureau has initiated efforts to acquire DRIS and FDCA, key
systems it needs to perform the 2010 Decennial Census. It awarded a
contract for DRIS in October 2005; the system is currently in a design
and development phase. The bureau expects to award a contract for FDCA
development in March 2006. While both projects‘ life cycle cost
estimates are currently considered procurement sensitive, together they
are expected to make up a large portion of the $1.8 billion program to
develop, test, and implement decennial census systems. Both
acquisitions involve ambitious schedules in order to be able to deliver
the needed functionality to support a planned 2008 census dress
rehearsal and the eventual 2010 census activities.
While both project offices have implemented initial acquisition
management activities, neither has the full set of capabilities they
need to effectively manage the acquisitions (see table below).
Specifically, the DRIS project completed its solicitation activities
and the FDCA project has completed most of its solicitation activities.
However, activities in other management areas have been initiated but
not completed. For example, the DRIS project office has established
baseline requirements for the acquisition, but the bureau has not yet
validated them or implemented a process for managing the requirements.
Also, while the FDCA project office has initiated efforts to oversee
the contractor‘s performance, such as requiring earned value management
reporting (a project management tool that integrates cost, schedule,
and scope of work to aid project planning and control) and hiring staff
with contracting experience, it has not yet determined which
performance measures it will use to track the contractor and its own
internal project office performance. Until these and other basic
contract management activities are fully implemented, both projects
face increased risks that the systems will experience cost overruns,
schedule delays, and performance shortfalls.
Status of Acquisition Management Activities for Key Decennial Systems:
[See PDF for image]
[End of figure]
What GAO Recommends:
GAO recommends that the Census Director ensure that the bureau
completes key activities needed to effectively manage its acquisitions.
In commenting on a draft of GAO‘s briefing, bureau officials generally
agreed with the recommendations and noted that time and budget
constraints had driven the bureau to proceed with its acquisitions
before all of the recommended activities had been completed. Officials
stated that they plan to complete these activities as soon as possible.
www.gao.gov/cgi-bin/getrpt?GAO-06-444T.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact David A. Powner at (202)
512-9286 or pownerd@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
We appreciate the opportunity to participate in today's hearing to
discuss our work on key acquisitions supporting the 2010 Decennial
Census. As you know, the Census Bureau is currently planning the
decennial census--the nation's oldest and most comprehensive source of
population and housing data. While apportionment--the proportional
distribution of the number of members of the U.S. House of
Representatives in each state on the basis of the population of each
state--is the most widely known use of census data, the data are also
used for congressional redistricting, managing federal agencies, and
allocating federal funds. These data are disseminated to state and
local governments, academia, and the private sector for use in
understanding this country's people and their social, demographic, and
economic characteristics. The next decennial census is required to
begin on April 1, 2010, and the Secretary of Commerce is required to
report to the President the tabulation of total population by states
within 9 months of that date.[Footnote 1]
In June 2005, we reported on the Census Bureau's progress in five
information technology (IT) management areas--investment management,
systems development/management, enterprise architecture management,
information security, and human capital.[Footnote 2] These IT
management areas are important because they have substantial influence
on the effectiveness of organizational operations and--if applied
effectively--can reduce the risk of cost and schedule overruns, and
performance shortfalls. We reported that, while the bureau had many
practices in place, much remained to be done to fully implement
effective IT management capabilities. We also made several
recommendations to improve the bureau's management. Given the
weaknesses we noted in the five management areas and the importance of
IT investments to the upcoming 2010 Decennial Census, you asked us to
review two planned Census Bureau acquisitions: the Decennial Response
Integration System (DRIS)--a system for integrating paper, Internet,
and telephone responses; and the Field Data Collection Automation
(FDCA) program--the systems, equipment, and infrastructure field staff
will use to collect census data.
In January, we presented a detailed briefing to the committee and
subcommittee staffs, which is provided in appendix I. At your request,
I will discuss key findings from that briefing, specifically covering
each project's status and management capabilities. An overview of the
approach we used to perform this work--our objectives, scope, and
methodology--is provided in appendix II.
Results in Brief:
The Census Bureau has initiated efforts to acquire DRIS and FDCA, key
systems it needs to perform the 2010 Decennial Census. It awarded a
contract for DRIS in October 2005; the system is currently in a design
and development phase. The bureau expects to award a contract for FDCA
development in March 2006. While both projects' life cycle cost
estimates are currently considered procurement sensitive, together they
are expected to make up a large portion of the $1.8 billion program to
develop, test, and implement decennial census systems. Both
acquisitions involve ambitious schedules in order to be able to deliver
the needed functionality to support a planned 2008 census dress
rehearsal and the eventual 2010 census activities.
While both project offices have implemented initial acquisition
management activities, neither has the full set of capabilities they
need to effectively manage the acquisitions. Specifically, the DRIS
project completed its solicitation activities and the FDCA project has
completed most of its solicitation activities. However, activities in
other management areas have been initiated but not completed. For
example, the DRIS project office has established baseline requirements
for the acquisition, but the bureau has not yet validated them or
implemented a process for managing the requirements. Also, the project
office identified project risks, but has not yet developed written
mitigation plans or established milestones for completing key risk
mitigation activities. Regarding FDCA, the project office has developed
baseline functional requirements for the acquisition, but the bureau
has not yet validated them. Also, while the FDCA project office has
initiated efforts to oversee the prospective contractor's performance,
such as requiring earned value management[Footnote 3] reporting and
hiring staff with contracting experience, it has not yet determined
which performance measures it will use to track the contractor and its
own internal project office performance. Additionally, the project
office identified risks, but it has not yet implemented a risk
management process. Specifically, it has not yet assigned
responsibilities, developed detailed mitigation plans for managing the
risks, or established milestones for completing key mitigation
activities.
Until these and other basic management activities are fully
implemented, both projects face increased risks that the systems will
experience cost overruns, schedule delays, and performance shortfalls.
We are making recommendations to the Director of the Census Bureau to
ensure that both project offices complete key activities needed to
effectively manage acquisitions, including activities associated with
effective project planning, requirements management, risk management,
and performance measurement.
Background:
The Census Bureau's mission is to serve as the leading source of
quality data about the nation's people and economy. While apportionment
is the most widely known use of census data, the data are also used for
congressional redistricting, managing federal agencies, allocating
federal funds, and they are disseminated to state and local
governments, academia, and the private sector. Data from a decennial
census provide official, uniform information gathered over the decades
on the country's people and their social, demographic, and economic
characteristics. They provide the baselines for countless other surveys
and are used to develop sampling frames for a number of other federal
data collections, such as the Current Population Survey, which is used
to measure participation in the labor market and unemployment
rates.[Footnote 4]
The bureau's decennial census organization, which is responsible for
the decennial census program, is comprised of several divisions and
offices. The Decennial Management Division is responsible for
implementing the decennial census. The Decennial Systems and Contracts
Management Office manages selected system contracts supporting the
decennial census.
To support the 2010 Decennial Census, the bureau manages a $1.8 billion
program called "2010 Testing, Evaluation, and Systems Design," which
calls for the acquisition and testing of systems and technologies. Two
of the key acquisitions associated with this program are DRIS and FDCA.
In addition, other key systems support the planned 2010 decennial
census. Together, these and other systems are to support the
collection, processing, and dissemination of census data. Figure 1
shows an overview of the key systems planned to support the 2010
Decennial Census.
Figure 1: Overview of Systems Supporting the 2010 Decennial Census:
[See PDF for image]
[End of figure]
To effectively manage major IT programs, organizations should use sound
acquisition and management processes to minimize risks and thereby
maximize chances for success. Such processes include project and
acquisition planning, solicitation, requirements development and
management, risk management, contract tracking and oversight/project
monitoring and control, quality assurance, and executive oversight (see
table 1 for a description of each process). Such processes have been
identified and endorsed by leading organizations such as the Software
Engineering Institute and the Chief Information Officer's Council, and
in our prior work analyzing best practices in industry and government.
Our work has shown that such processes are significant factors in
successful systems acquisitions and development programs, and they
improve the likelihood of meeting cost and schedule estimates as well
as performance.
Table 1: Selected Processes for Effectively Managing IT Programs:
Process area: Project and acquisition planning;
Description: Effective project planning involves establishing and
maintaining plans defining project scope and activities, including
overall budget and schedule, key deliverables, and milestones for key
deliverables. It also involves ensuring that the project team has the
skills and knowledge needed to manage the project and obtaining
stakeholder commitment to the project plan. Effective acquisition
planning involves developing an acquisition strategy that includes
objectives, projected costs and schedules, and risks.
Process area: Solicitation;
Description: This activity includes identifying the needs within a
particular acquisition, developing and implementing a solicitation
plan, preparing for the evaluation of responses, issuing a solicitation
package, evaluating responses, conducting supporting negotiations, and
making recommendations for award of the contract.
Process area: Requirements development and management;
Description: Requirements development involves eliciting, analyzing,
and validating customer and stakeholder needs and expectations.
Requirements management involves establishing an agreed-upon set of
requirements, ensuring traceability between operational and product
requirements, and managing any changes to the requirements in
collaboration with stakeholders.
Process area: Risk management;
Description: An effective risk management process identifies potential
problems before they occur, so that risk-handling activities may be
planned and invoked as needed across the life of the product and
project in order to mitigate adverse impacts on achieving objectives.
Key activities include identifying and analyzing risks, assigning
resources, developing risk mitigation plans and milestones for key
mitigation deliverables, briefing senior-level managers on high-
priority risks, and tracking risks to closure.
Process area: Contract tracking and oversight/project monitoring and
control;
Description: These processes provide oversight of the contractor's and
the project office's performance, in order to allow appropriate
corrective actions if actual performance deviates significantly from
the plan. Key activities in tracking both the contractor's and the
project office's performance include the selection of performance
measures, communicating status, taking corrective actions, and
determining progress.
Process area: Process and product quality assurance;
Description: This process area provides staff and management with
objective insight into processes and associated work products. This
includes the objective evaluation of project processes and products
against approved descriptions and standards. Key activities include
developing a quality assurance plan, assigning resources to quality
assurance activities, and implementing quality assurance activities.
Through quality assurance, the project team is able to identify and
document noncompliance issues and provide appropriate feedback to
project staff.
Process area: Executive oversight and governance;
Description: Office of Management and Budget and GAO guidance call for
agencies to establish IT investment management boards comprised of key
executives to regularly track the progress of major systems
acquisitions. These boards should have documented policies and
procedures for management oversight of IT projects and systems, and
should be able to adequately oversee the project's progress toward cost
and schedule milestones and their risks. The board should also employ
early warning systems that enable it to take corrective actions at the
first sign of cost, schedule, and performance slippages.
Source: GAO summary of leading practices, including practices
identified by the Software Engineering Institute, the Chief Information
Officer's Council, and the Office and Management and Budget.
[End of table]
DRIS Project Under Way but Key Management Activities Remain To Be
Implemented:
DRIS is intended to receive, capture, and standardize census data
provided by respondents via census forms, telephone agents, and the
Internet. DRIS is also intended to standardize data collected via
mobile computing devices and perform other functions, such as provide
assistance to the public via telephone and the Internet.
On October 5, 2005, the bureau awarded a cost-plus-award-fee contract
for DRIS to Lockheed Martin Corporation and its seven subcontractors to
design, develop, and implement a system for standardizing and
integrating the data from all of the response modes. The contract has a
6-year performance period, which includes designing and developing the
system, conducting the census dress rehearsal in 2008, conducting 2010
census operations, and archiving data and disposing of equipment after
the census is completed. After the contract was awarded, the project
got off to a slow start because of a bid protest that has since been
withdrawn. The DRIS project office is currently reassessing the project
schedule due to delays associated with the bid protest. The project
office plans to complete this assessment and perform an integrated
baseline review by March 2006.[Footnote 5]
The DRIS project office has initiated activities supporting key project
management processes, but does not yet have the full set of acquisition
management capabilities it needs to effectively manage the acquisition.
Table 2 provides a summary of the status of the DRIS project in each of
the process areas we evaluated.
Table 2: Summary of the Status of DRIS Acquisition Management
Capabilities:
Capability: Project and acquisition planning;
Status: Initiated, but key activities remain to be completed, such as
finalizing the project management plan and identifying key deliverables
beyond 2008.
Capability: Solicitation;
Status: Key activities completed and contract awarded ahead of
schedule.
Capability: Requirements development and management;
Status: Initiated, but key activities remain to be completed, such as
validating requirements and implementing a requirements management
process.
Capability: Risk management;
Status: Initiated, but key activities remain to be performed such as
preparing mitigation plans and holding senior-level briefings.
Capability: Contract tracking and oversight/project monitoring and
control;
Status: Initiated, but project office performance measures have not yet
been established.
Capability: Process and product quality assurance;
Status: Initiated, but a quality assurance process is not yet in place.
Capability: Executive oversight/governance;
Status: Initiated, but key governance activities remain to be
completed.
Source: GAO analysis of Census Bureau data.
[End of table]
The DRIS project office has made progress in building management
capabilities, but more remains to be done in key areas. For example,
the project office completed its solicitation activities and awarded
the contract ahead of schedule. In the area of requirements development
and management, the project office established basic functional
requirements for the acquisition and the Decennial Management Division
has developed a detailed change control process for managing
requirements for all 2010 Decennial Census systems, including DRIS.
However, the division has not yet conducted a thorough validation of
DRIS requirements, ensured traceability between DRIS and operational
requirements, or approved the DRIS baseline requirements. Further, the
division has not yet staffed the teams responsible for managing
requirements or initiated any requirements management activities.
Until the bureau obtains validation and approval of DRIS requirements,
it faces increased risk that new and changing requirements could be
identified throughout the system's development. Changes identified late
in the development cycle could be costly to implement and could
increase the risk that the system will not be ready in time to support
census activities. Further, until the bureau staffs and implements its
planned requirements management process, it may not be able to
effectively ensure that resulting products meet requirements.
In the area of risk management, the DRIS project office has developed a
pre-award risk management plan that defines staff roles and
responsibilities and includes procedures for identifying and tracking
risks and risk mitigation actions. Since awarding the contract, the
team has updated the pre-award risks. However, the project team has not
written mitigation steps or dates associated with the team's plans for
addressing high-impact risks, and the project team has not conducted
senior-level briefings. Until the project team develops risk mitigation
plans with milestones for key activities and regularly briefs senior-
level managers on risks and risk mitigation plans, it faces increased
probability that DRIS will not be delivered on schedule, within budget,
or perform as expected.
In the area of contract and project monitoring, the DRIS project team
plans to provide contract tracking and oversight through reports
including earned value management and monthly status reports. However,
the project office has not yet selected detailed performance measures
for tracking the contractor or its own internal progress (such as
progress against planned milestones, number of risks opened and closed,
number and frequency of changes to requirements, and frequency of
quality assurance reviews). As a result, the project team is not able
to ensure that it will be able to identify and document any
noncompliance issues and take appropriate corrective actions.
One of the root causes of the project's delay in completing key
management activities is that the Census Bureau lacks the
organizational policies for managing major acquisitions. As a result,
the success of major acquisitions such as DRIS is highly dependent on
the knowledge, skills, and qualifications of the project offices.
Without a minimum set of required steps and processes, Census
acquisitions are at increased risk of being run in an ad hoc and
chaotic manner--potentially resulting in increased project costs,
delayed schedules, and performance shortfalls. In commenting on a draft
of our briefing, agency officials acknowledged that the bureau does not
have an organizational acquisition management policy, but noted that,
even if there were a policy, time and budget constraints have driven
the bureau to proceed with its acquisitions before all of the
recommended activities have been completed. Officials stated that they
plan to complete these activities as soon as possible.
FDCA Program Has Been Initiated but Key Management Activities Remain To
Be Performed:
The FDCA program is expected to provide automation support for field
data collection operations for the 2010 Census. The program is expected
to provide office automation for regional and local census offices; the
telecommunications infrastructure for headquarters, regional, and local
offices; and mobile computing devices for field workers. The bureau
plans to have field-based enumerators use nearly 500,000 mobile
computing devices to support decennial census field operations. Our
companion testimony provides details on mobile computing devices and
concerns about the reliability observed during testing.[Footnote 6]
Enumerators from local census offices will use these mobile computing
devices to complete activities including address canvassing (visiting
households to update address lists and collect global positioning
coordinates to update maps) and conducting non-response follow-up
(visiting households to obtain information from households that do not
provide responses via mail, Internet, or phone). The bureau plans to
award the FDCA contract, which is expected to be a cost-reimbursement
contract with multiple incentives, on March 31, 2006. The contract is
expected to have four phases--one planning and three execution phases.
The FDCA project office has initiated many key processes to oversee and
manage the contract, but has not yet implemented the full set of
acquisition management capabilities it needs to fully manage the
acquisition. Table 3 provides a summary of the status of the FDCA
project in each of the process areas we evaluated.
Table 3: Summary of the Status of FDCA Acquisition Management
Capabilities:
Capability: Project and acquisition planning;
Status: Initiated, but key activities remain to be completed, such as
identifying deliverables and milestones, and obtaining stakeholder buy-
in on a project plan that includes the project's estimated costs,
budget, and schedules.
Capability: Solicitation;
Status: Key activities completed; bureau officials said that they are
on schedule to award the contract in March 2006.
Capability: Requirements development and management;
Status: Initiated, but key activities remain to be completed such as
validating requirements.
Capability: Risk management;
Status: Initiated, but key activities remain to be performed such as
assigning responsibilities for risks and preparing mitigation plans.
Capability: Contract tracking and oversight/project monitoring and
control;
Status: Initiated, but key performance measures have not yet been
established; bureau officials said that they expect to define these
metrics after contract award.
Capability: Process and product quality assurance;
Status: Initiated; the quality assurance process is expected to be
implemented after contract award.
Capability: Executive oversight/governance;
Status: Initiated, but key governance activities remain to be
completed.
Source: GAO analysis of Census Bureau data.
[End of table]
The FDCA project office has made progress in building management
capabilities, but more remains to be done in key areas. For example,
the project office completed many key solicitation activities and plans
to award a contract in March 2006. In the requirements development and
management area, the project office and the Decennial Management
Division developed FDCA-specific functional requirements with
participation from stakeholders. The office has also drafted a
requirements management plan. However, the division has not yet
validated and approved a baseline set of operational requirements or
ensured traceability between its operational requirements and the FDCA
request for proposal. Until the bureau finalizes its operational
requirements for the census and ensures that the FDCA request for
proposal is consistent with the baseline requirements, the project will
be at risk of having changes to the requirements, potentially affecting
its ambitious development and implementation schedule.
In the risk management area, the project office has developed a draft
risk management process and identified a number of high-level risks for
the FDCA project. Also, to manage its schedule and technical risks, the
project office has adopted an approach calling for pre-award prototype
development. However, the FDCA project office has not yet revisited or
analyzed the identified risks, begun prioritizing and tracking project
risks, or documented risk mitigation plans. Until the team implements
an effective risk management process, it will lack a mechanism to
address known and unknown problems.
Additionally, in the contract and project monitoring area, the project
office has initiated efforts to oversee the contractor's performance,
such as requiring earned value management reporting and hiring staff
with contract tracking and oversight experience. However, the project
office has not yet selected detailed performance measures for tracking
the contractor or its own internal progress (such as progress against
planned milestones, number of risks opened and closed, number and
frequency of changes to requirements, and frequency of quality
assurance reviews). Without such practices in place, the project team
is not able to ensure that it will be able to identify problems and
take appropriate corrective actions in a timely manner.
Conclusions:
While the DRIS and FDCA project offices have initiated important steps
in establishing sound and capable project management, neither has
completed all of the important activities needed to adequately manage
the acquisitions. Incomplete management activities, including those for
requirements management, risk management, and contract and project
monitoring, increase the risk that these projects will encounter
problems in meeting cost and schedule expectations. Given the immovable
deadline for performing the 2010 Decennial Census, if unexpected
problems or changes occur, it is more likely that the bureau will face
cost overruns or be forced to accept a system with limited
functionality. Since the DRIS contract was awarded in October 2005, and
the FDCA contract is expected to be awarded in March, it is critical
that the DRIS project office expeditiously put in place key elements of
sound acquisition management capability. Bureau officials acknowledge
the importance of implementing these acquisition management processes
and state that they plan do to so as soon as possible.
Recommendations for Executive Action:
To ensure that the bureau adequately manages the DRIS project, we
recommend that the Director of the Census Bureau direct the project
office to take the following six actions:
* Complete the DRIS project plan and obtain stakeholders' commitment to
the plan before initiating further development work.
* Obtain validation, management, and customer approval of DRIS
requirements.
* Staff teams and perform planned requirements management activities.
* Develop mitigation plans with milestones for key activities, and
regularly brief senior managers on important risks.
* Establish performance measures and monitor results for contractor and
project office activities.
* Implement a quality assurance process by establishing
responsibilities for assuring product quality and defining how
inspections, reviews, and walkthroughs will be used to measure quality.
Further, to ensure that the bureau improves its ability to manage this
and other acquisitions, we recommend that the Director of the Census
Bureau immediately establish and enforce a system acquisition
management policy that incorporates best practices in system and
software acquisition management (including those highlighted in our
briefing).
To ensure that the bureau adequately manages the FDCA project, we
recommend that the Director of the Census Bureau direct the project
office to take the following four actions:
* Obtain stakeholder commitment to a project plan that includes
estimated project costs and schedules, including deliverables and
milestones.
* Obtain validation and approval of baseline requirements.
* Identify, prioritize, and assign responsibilities for risks, and
develop and implement risk mitigation plans and actions.
* Develop internal and contractor performance measures and prepare to
track project cost, schedule, and performance.
Agency Comments and Our Evaluation:
In providing oral and e-mail comments on a draft of our briefing,
Census Bureau officials--including the Associate Director for the
Decennial Census and the Assistant Director for Decennial Information
Technology and Geographic Systems--generally agreed with our
recommendations and stated that time constraints and budgets have
driven the bureau to proceed with its acquisitions before all of the
recommended activities have been completed. Officials stated that they
plan to complete these activities as soon as possible.
Bureau officials also stated that they intend to rely on the DRIS and
FDCA contractors to help refine requirements, project plans, and
performance measures. However, our experience in reviewing major system
acquisitions in recent years has shown that there are risks associated
with relying too heavily on contractors to perform key management and
oversight activities. For example, after a long history of significant
cost increases and schedule delays on its Business System Modernization
program, the Internal Revenue Service recently began transferring
responsibility for key program management operations (including cost
and schedule estimation and measurement, integration testing, and risk
management) away from its contractor and back to the agency because of
the contractor's poor performance in these areas.[Footnote 7] Clearly,
it is important for the government to exercise strong leadership in
managing requirements, plans, risks, and performance measures. Bureau
officials also offered technical corrections, which we incorporated in
the briefing and in this statement as appropriate.
This concludes my statement. I would be pleased to respond to any
questions that you or other members of the Subcommittee may have at
this time.
Contacts and Acknowledgements:
If you have any questions regarding this testimony, please contact
David Powner at (202) 512-9286 or by e-mail at [Hyperlink,
pownerd@gao.gov]. Individuals making contributions to this testimony
include Neil Doherty, Amanda Gill, Nancy Glover, Colleen Phillips, and
Cynthia Scott.
[End of section]
Appendix I: GAO Briefing to Committee and Subcommittee Staffs on
January 30, 2006:
Census Bureau:
Key Acquisitions Supporting the 2010 Decennial Census Face Challenges:
Briefing for the Committee on Government Reform and its Subcommittee on
Federalism and the Census:
House of Representatives:
January 30, 2006:
Briefing Outline:
Introduction:
Objectives:
Scope and Methodology:
Results in Brief:
Background:
Decennial Response Integration System:
* Overview, Status, and Plans:
* Acquisition Management Capabilities:
* Conclusions:
* Recommendations:
Field Data Collection Automation program:
* Overview, Status, and Plans:
* Acquisition Management Capabilities:
* Conclusions and Recommendations:
Agency Comments and Our Evaluation:
Introduction:
In June 2005, we reported on the Census Bureau's progress in five
information technology (IT) management areas-investment management,
systems development/management, enterprise architecture management,
information security, and human capital. [NOTE 1] We reported that,
while the bureau had many practices in place, much remained to be done
to fully implement effective IT management capabilities. We made
several recommendations intended to improve these capabilities across
the bureau.
Given the weaknesses we noted in the five IT management areas and the
importance of IT investments to the upcoming 2010 Decennial Census, the
Chairman of the House Committee on Government Reform and the Chairman
and Ranking Member of the Subcommittee on Federalism and the Census
requested that we assess the bureau's ability to manage important new
acquisitions intended to support the decennial census.
NOTE:
[1] GAO, Information Technology Management. Census Bureau Has
Implemented Many Key Practices, but Additional Actions Are Needed, GAO-
05-661 (Washington, D.C.: June 16, 2005).
Objectives:
As part of the Census Bureau's plans to increase the use of automation
and technology for the 2010 Decennial Census, the bureau will be
undertaking several major acquisitions including the Decennial Response
Integration System (DRIS)-a system for integrating paper, Internet, and
telephone responses, and the Field Data Collection Automation (FDCA)
program-the systems and support equipment for field office data
collection activities.
Our objectives for each of these acquisitions are to:
* provide an overview, status, and plans (including costs, schedule,
and deliverables) and:
* determine if the bureau has capabilities in place to successfully
manage and oversee the acquisitions.
Scope and Methodology:
To provide the project's overview, status, and plans, we analyzed
current project documents including plans, acquisition documents, and
deliverables, and we interviewed project officials.
To determine if the bureau has capabilities in place to successfully
manage and oversee the acquisitions, we identified sound IT systems
acquisition and management processes from industry standards, including
those developed by the Software Engineering Institute, and compared
them to the Census Bureau's practices for the selected acquisitions. We
evaluated the following processes:
* project and acquisition planning;
* solicitation;
* requirements development and management;
* risk management;
* contract tracking and oversight/project monitoring and control;
* process and product quality assurance;
* executive oversight and governance.
In each process area, we evaluated project documentation and
interviewed project officials to determine the status of the bureau's
efforts.
We conducted our review between July 2005 and January 2006 at Census
Bureau headquarters in Suitland, Maryland, in accordance with generally
accepted government auditing standards.
Results in Brief:
The Census Bureau has initiated efforts to acquire the major systems it
needs to perform the 2010 Decennial Census, including the Decennial
Response Integration System (DRIS), and the Field Data Collection
Automation (FDCA) program. It awarded a contract for DRIS in October
2005 and the system is currently in a design and development phase. The
bureau expects to award a contract for FDCA development in March 2006.
While both projects' life cycle cost estimates are currently considered
procurement sensitive, together they are expected to make up a large
portion of the $1.8 billion program to develop, test, and implement
decennial census systems. Both acquisitions involve ambitious schedules
in order to be able to deliver the needed functionality to support a
planned 2008 census dress rehearsal and the eventual 2010 census
activities.
While both projects have implemented initial project management
activities, neither has the full set of acquisition management
capabilities they need to fully manage the acquisitions. For example,
the DRIS project office has a draft project plan that contains many of
the needed elements, but it has not yet completed the plan or obtained
written stakeholder buy-in on it. It established baseline requirements
for the acquisition, but the bureau has not yet validated them or
implemented a process for managing the requirements. Also, the project
office identified risks, but has not yet implemented a risk management
process. Specifically, the project office has not yet developed written
mitigation plans or established milestones for completing key risk
mitigation activities.
Regarding FDCA, its project office has initiated a project plan, but
has not yet obtained written stakeholder buy-in on an overall plan that
includes the project's estimated costs, budget, and schedules. It has
identified requirements for the acquisition, but the bureau has not yet
approved a validated set of requirements or ensured that the
acquisition requirements are traceable to the broader set of
operational requirements for the decennial census. Further, while the
project office identified risks, it has not yet assigned
responsibilities or developed detailed mitigation plans for managing
the risks, or established milestones for completing key mitigation
activities.
Until these and other basic management capabilities are fully
implemented, both projects face increased risks that the systems will
experience cost overruns, schedule delays, and performance shortfalls.
We are making recommendations to both project offices and to the bureau
to help improve acquisition management capabilities.
In commenting on a draft of this briefing, Census Bureau officials,
including the Associate Director for the Decennial Census and the
Assistant Director for Decennial Information Technology and Geographic
Systems, generally agreed with our recommendations and stated that time
constraints and budgets have driven the bureau to proceed with its
acquisitions before all of the recommended activities have been
completed. Officials noted that they plan to complete these activities
as soon as possible. They also offered technical corrections, which we
have incorporated as appropriate.
Background:
Census Bureau Mission and Core Activities:
The Census bureau's mission is to serve as the leading source of
quality data about the nation's people and economy. The bureau's core
activities include:
* conducting decennial, economic, and government censuses,
* conducting demographic and economic surveys,
* managing international demographic and socioeconomic databases and
providing technical advisory services to foreign governments, and
* performing other activities such as producing official population
estimates and projections.
Public and private decision makers use census population and
socioeconomic data for various purposes. For example:
* decennial census data are used to determine congressional and state
legislative districts and to distribute hundreds of billions of dollars
in federal funds each year,
* federal agencies use census data to evaluate the effectiveness of
established programs, and,
* businesses use census data to target new services and products and to
tailor existing ones to demographic changes.
Census Bureau Organization:
The bureau is a large and complex organization. A conceptual view of
the agency includes three core organizations, two auxiliary
organizations that provide guidance and operational support for the
core organizations, and three support organizations that provide
administrative and technical support for the entire bureau. Each of
these organizations is headed by an associate director, who reports to
the deputy director of the Census Bureau.
Census Bureau:
[See PDF for image]
Source: U.S. Census Bureau, GAO.
[End of figure]
The Bureau's Decennial Census:
The bureau's Decennial Census organization is responsible for the
decennial census program, the nation's oldest and most comprehensive
source of population and housing information.
Conducting a decennial census involves:
* identifying and correcting addresses for all known living quarters in
the United States,
* sending questionnaires to housing units,
* following up with non-respondents through personal interviews,
* trying to identify people with non-traditional living arrangements,
* managing a voluminous workforce responsible for follow-up activities,
* collecting census data from questionnaires, calls, and personal
interviews,
* summarizing and tabulating census data, and:
* disseminating census analytical results to the public.
The Decennial Census organization is comprised of divisions and
offices, including the Decennial Management Division (DMD), which is
responsible for implementing the decennial census, and the Decennial
Systems and Contracts Management Office, which manages selected system
contracts supporting the decennial census.
Plans for the 2010 Decennial Systems:
To support the 2010 Decennial Census, the bureau manages a $1.8 billion
program called "2010 Testing, Evaluation, and Systems Design," which
calls for the acquisition and testing of systems and technologies. Two
of the key acquisitions are the Decennial Response Integration System
(DRIS) and the Field Data Collection Automation (FDCA) program.
DRIS is expected to be a system for collecting data and integrating
census responses that come in through multiple routes, including census
forms, telephone agents, the Internet, and from mobile computing
devices used by field staff. DRIS is expected to standardize the
response data and to provide it to other bureau systems for analysis
and processing.
The FDCA program is expected to provide mobile computing devices,
office automation, and an IT infrastructure to support collection of
census data in local and regional offices. Mobile computing devices
will be used to update the bureau's address list, to perform follow up
at addresses for which no questionnaire was returned, and to perform
activities to measure census coverage.
Plans for the 2010 Decennial Systems:
In addition to DRIS and FDCA, other key systems support the planned
2010 decennial census. These include:
* master address file/topologically integrated geographic encoding and
referencing-this is an existing system that contains the master list of
addresses for the census,
* headquarters data processing systems-these are existing systems used
to process census data as well as a management information system,
* data access and dissemination system II-this is a planned upgrade to
an existing system to disseminate 2010 census data to the public.
Together, these systems are to support the collection, processing, and
dissemination of census data, as illustrated on the next page.
Overview of systems supporting the 2010 Decennial Census:
[See PDF for image]
Source: GAO analysis of Census data.
[End of figure]
The bureau plans a series of tests in the years leading up to the
decennial census.
2004: The bureau tested critical field operations using systems under
conditions similar to those to be used during the decennial census. In
particular, the agency studied the feasibility of using hand-held
mobile computing devices equipped with Global Positioning System
capability to conduct non-response follow-up operations. We recently
reported on lessons learned during this test. [NOTE 2]
2006: The bureau plans to test the methodology and functions of the
integration of systems needed to carry out the census, focusing on
efforts to automate non-response follow-up activities and the
initiatives to update the address list.
2008: The bureau plans to conduct a final operational test of the
entire complement of methodological, procedural, and systems
innovations for the 2010 Decennial Census. This test is known as the
Dress Rehearsal.
NOTE:
[2] GAO, 2010 CENSUS: Basic Design Has Potential, but Remaining
Challenges Need Prompt Resolution, GAO-05-9 (Washington, D.C.: Jan. 12,
2005).
DRIS:
Overview, Status, and Plans:
During the prior decennial census, called Census 2000, responsibilities
for designing and developing a system for capturing census data and
related functions were shared by the bureau and several contractors.
The bureau designed and developed a system for collecting data from the
Internet, while one contractor developed a system for providing
telephone assistance to the public, and another contractor developed a
system for capturing data from responses returned by mail and from the
field operations targeting non-responders. Another contractor provided
the staffing and facilities for operating the data capture system while
the census was underway. The bureau then integrated all of the data.
Subsequently, the bureau found that this diversified approach resulted
in data that was not standardized and added to the cost of processing
census data.
For the 2010 Decennial Census, the bureau plans to have a single
contractor design, develop, and implement a system (DRIS) for
standardizing and integrating the data from all of the response modes
(paper, telephone, Internet, and from field operations). [NOTE 3] DRIS
is intended to:
* receive, capture, and standardize census data provided by respondents
via census forms, telephone agents, and the Internet,
* standardize data collected via mobile computing devices,
* provide the data to the headquarters data processing system,
* provide assistance to the public via telephone and the Internet, and
* monitor the quality and status of data capture operations.
DRIS is expected to process an estimated 90 million mailed paper
responses, 40 million field responses, 9 million telephone calls, and 9
million Internet visits.
According to the bureau, qualitative improvements expected from DRIS
include:
* integrating paper, Internet, and telephone responses within a
workflow control system that will provide more timely information than
existed in previous censuses and reduce the number of cases of non-
response follow-up,
* providing near real-time data that will support the planned second
mailing to non-responding addresses, and:
* reducing redundant efforts for the Census Bureau and contractors that
existed for different contracts during Census 2000 by integrating key
functions under one contract. In particular, efforts in the areas of
security, data integration, change control, and contractor
administration are expected to be reduced.
The bureau began acquisition planning for DRIS in 2003. Between 2003
and September 2005, the bureau spent about $7.5 million on researching
data capture technologies, conducting web-based vendor briefings,
obtaining comments from prospective vendors, and developing DRIS
planning documents, strategies, and analyses. During this time, the
bureau also established the DRIS project office within the Decennial
Systems and Contracts Management Office (DSCMO).
Further, between February and August 2005, the bureau completed
solicitation activities as follows:
* In February 2005, the bureau issued a request for proposal (RFP) for
DRIS.
* From March 2005 through August 2005, the bureau solicitation team
reviewed and evaluated proposals and obtained oral presentations from
vendors.
* In August 2005, the solicitation team made a presentation to the
Source Selection Evaluation Board and recommended a vendor for contract
award.
On October 5, 2005 the bureau awarded the contract for DRIS to Lockheed
Martin Corporation and its 7 subcontractors. The DRIS performance
contract is a cost-plus-award-fee contract. The contract has a 6-year
performance period divided into 3 primary phases as follows:
Phase I: October 2005 through December 2008:
Includes delivering detailed design documentation, developing and
testing DRIS, conducting the 2008 dress rehearsal, and identifying data
centers for the 2010 Census.
Phase II: August 2008 through the end of January 2011:
Includes opening data center sites, completing operational testing,
conducting 2010 Census operations, and closing down the DRIS 2010
operations facilities once the census is complete.
Phase III: July 2010 through end of contract:
Includes archiving DRIS data and images in accordance with National
Archives and Records Administration guidelines and disposing of DRIS
equipment once it is no longer needed.
Phases I and II of this contract are valued up to $553 million. The
total life cycle cost of the project is considered procurement
sensitive pending the pricing of phase III.
Phase I of the contract involves the following key activities:
Activity: Perform integrated baseline review[NOTE 4];
Due Date: March 2006.
Activity: Deliver DRIS acquisition planning documents;
Due Date: February-October 2006.
Activity: Certify DRIS as ready for use at all 2008 dress rehearsal
facilities;
Due Date: December 2007.
Activity: Deliver proposal for phase II-system operations and
facilities;
Due Date: September 2008.
Activity: Complete dress rehearsal and production activities;
Due Date: December 2008.
[End of table]
The DRIS project team is currently reassessing these phase I dates due
to delays associated with a bid protest that has since been withdrawn.
It expects to identify new dates by March 2006. These new dates will
likely result in the first modification to the contract.
Also, the team expects to work with the contractor during phase I to
develop the milestones and deliverables for phases II and III.
NOTES:
[3] While the DRIS contractor is expected to standardize and organize
response data from the hand-held computers, the scope of the DRIS
contract does not include providing the systems or staff used for field
enumeration operations.
[4] An integrated baseline review is a joint assessment by the
contractor and the project team of the technical plans for a work
segment as well as the adequacy of the budgets, resources, and
schedules estimated to complete that work. This review results in a
detailed plan for work activities, costs, and schedules that is used as
the basis for tracking the earned value of the contractor's
deliverables.
DRIS:
Acquisition Management Capabilities:
To effectively manage major IT programs, organizations use sound
acquisition and management processes to minimize risks and thereby
maximize chances for success. Such processes have been identified by
leading organizations such as the Software Engineering Institute, the
Chief Information Officer's Council, and in our prior work analyzing
best practices in industry and government. Key areas include:
* project and acquisition planning,
* contract tracking and oversight/project monitoring and control,
* solicitation,
* requirements development and management,
* process and product quality assurance,
* risk management,
* executive oversight and governance.
Our work has shown that such processes are significant factors in
successful systems acquisitions and development programs and that they
improve the likelihood of meeting cost and schedule estimates as well
as performance requirements.
DRIS:
Summary-Acquisition Management Capabilities:
Summary of the status of DRIS acquisition management capabilities.
[See PDF for image]
[End of table]
Project and acquisition planning:
Effective project planning involves establishing and maintaining plans
defining project scope and activities, including overall budget and
schedule, key deliverables, and milestones for key deliverables. It
also involves ensuring that the project team has the skills and
knowledge needed to manage the project and obtaining stakeholder
commitment to the project plan. Effective acquisition planning involves
developing an acquisition strategy that includes objectives, projected
costs and schedules, and risks.
The DRIS project team has:
* defined the overall project scope, budget, and schedule,
* developed a draft project management plan that identifies key
deliverables and milestones for these deliverables through the 2008
dress rehearsal,
* assigned an experienced and certified project manager and project
team, and:
* developed an acquisition strategy that outlines the acquisition
objectives, scope, costs, schedules, and risks.
However, the project team has not yet finalized its project plan,
identified key deliverables beyond 2008, or obtained stakeholder
commitment to the plan. The project team has not yet completed these
activities in part because there is no Census Bureau policy requiring
them to do so prior to contract award.
Until a project plan is completed and approved, the project lacks
assurance that it is moving in the right direction. Without this
assurance, it is more likely to encounter unanticipated changes in
direction-which could affect system cost, schedule, and deliverables.
Solicitation:
This activity includes identifying the needs within a particular
acquisition, developing and implementing a solicitation plan, preparing
for the evaluation of responses, issuing a solicitation package,
evaluating responses, conducting supporting negotiations, and making
recommendations for award of the contract.
The DRIS project team has:
* identified the needs for the DRIS acquisition including telephone,
Internet, and paper data capture modes,
* developed source selection procedures,
* developed the evaluation criteria for evaluating vendors' proposals
(including criteria for the technical and program management
approaches),
* conducted training on the evaluation process and on using an
automated tool, and submitted a recommendation for contract award to
the source selection officer in August 2005.
The contract was awarded in October 2005.
Requirements development and management:
Requirements development involves eliciting, analyzing, and validating
customer and stakeholder needs and expectations. Requirements
management involves establishing an agreed-upon set of requirements,
ensuring traceability between operational and product requirements, and
managing any changes to the requirements in collaboration with
stakeholders.
To develop and manage DRIS requirements:
* DMD, the organization that oversees the project as well as the
overall decennial census, developed draft operational requirements for
the overall decennial census,
* the project team established basic functional requirements for the
DRIS acquisition based on the data capture activities used during
Census 2000 and assumptions for the 2010 Census data capture needs,
* stakeholders reviewed the DRIS requirements and made suggestions for
clarifying them, and:
* DMD developed a detailed process for managing requirements for all
2010 decennial systems, including DRIS; these are to be executed by DMD
implementation teams.
However, DMD has not yet conducted a thorough validation of DRIS
requirements, ensured traceability between DRIS and operational
requirements, or approved the DRIS baseline requirements. Part of the
reason that these activities have not been completed is that there is
no Census Bureau policy requiring them to do so prior to contract
award. Further, DMD has not yet staffed the teams responsible for
managing requirements or initiated any requirements management
activities.
Until the project team obtains validation and approval of DRIS
requirements and ensures these requirements are traceable to the
operational requirements, it faces increased risk that new and changing
requirements could be identified throughout the system's development.
[NOTE 5] Changes identified late in the development cycle could be
costly to implement and could increase the risk that the system will
not be ready in time to support census activities.
Further, until the bureau staffs and implements its planned
requirements change management process, it may not be able to
effectively ensure that resulting products meet requirements. As a
result, DRIS may not provide the functionality needed or the bureau may
experience cost increases and schedule delays.
Risk Management:
An effective risk management process identifies potential problems
before they occur, so that risk-handling activities may be planned and
invoked as needed across the life of the product and project in order
to mitigate adverse impacts on achieving objectives. Key activities
include identifying and analyzing risks, assigning resources,
developing risk mitigation plans and milestones for key mitigation
deliverables, briefing senior-level managers on high priority risks,
and tracking risks to closure.
The DRIS project team has:
* developed a pre-award risk management plan that defines staff roles
and responsibilities and includes procedures for identifying and
tracking risks and risk mitigation actions,
* identified pre-award risks including:
- insufficient project office or contracting office resources;
- a bid protest;
- inadequate funding;
- failure to obtain agreement on quality control operations;
- failure to mitigate privacy risks;
- inadequate system sizing and related testing;
- design not flexible enough to accommodate changes in technology
related to security;
- failure to document and test external interfaces.
* assigned responsibilities for managing the risks and discussed
mitigating actions, and
* participated in monthly meetings to analyze and discuss the status of
certain pre-award risks.
Since contract award in October 2005, the project team has:
* assigned a risk manager to provide oversight for the risk program,
* implemented a tool for tracking risks and actions taken, and:
* reviewed risks and updated the risk database.
Also, the project team plans to:
* conduct monthly internal reviews of DRIS risks and refer high risks
to higher-level officials for input and approval of actions,
* participate in the contractor's risk reviews and monitor the
contractor's risk management plans as part of the contract's
surveillance process, and:
* implement a process to assess risks based on the probability of
occurrence and the impact on business drivers.
However, in the months since the contract was awarded, the project team
has not developed written risk mitigation plans, identified milestones
for key mitigation activities, or briefed senior management on its
risks and risk mitigation plans. Specifically, there are no written
mitigation steps or dates associated with the agencies' plans for
addressing high-impact risks, and senior-level briefings on these risks
have not been held. Part of the reason that these activities have not
been completed is that there is no organizational policy requiring
completion of these activities.
Until the project team develops risk mitigation plans with milestones
for key activities, and regularly briefs senior-level managers on risks
and risk mitigation plans, it faces increased probability that DRIS
will not be delivered on schedule, within budget, or perform as
expected.
Contract tracking and oversight/project monitoring and control:
These processes provide oversight of the contractor's and the project
office's performance, in order to allow appropriate corrective actions
if actual performance deviates significantly from the plan. Key
activities in tracking both the contractor's and the project office's
performance include the selection of performance measures,
communicating status, taking corrective actions, and determining
progress.
The project team is responsible for ensuring that the contractor's
performance stays within cost and schedule thresholds for each phase of
the DRIS contract. Therefore, the project team plans to provide
contract tracking and oversight through:
* earned value management reports,
* contractor performance reports,
* program management reviews,
* monthly status reports.
During Phase 1, the project team plans to determine the detailed
requirements for these activities.
Contract and Project Monitoring and Control (continued):
However, the project office has not yet selected detailed performance
measures for tracking the contractor or its own internal progress (such
as progress against planned milestones, number of risks opened and
closed, number and frequency of changes to requirements, and frequency
of quality assurance reviews), in part because there is no Census
policy requiring that these measures be implemented. Without such
performance measures, the team cannot perform the other key activities
of communicating status, taking corrective actions, and determining
progress.
As a result, the project office's view into when performance deviates
from plans or when key activities are showing troubling trends is
limited. This can lead to unexpected cost increases, schedule delays,
and performance shortfalls.
Process and Product Quality Assurance:
This process area provides staff and management with objective insight
into processes and associated work products. This includes the
objective evaluation of project processes and products against approved
descriptions and standards. Key activities include developing a quality
assurance plan, assigning resources to quality assurance activities,
and implementing quality assurance activities. Through quality
assurance, the project team is able to identify and document
noncompliance issues and provide appropriate feedback to project staff.
The DRIS project team has identified several approaches that it plans
to use to oversee the quality of deliverables and services produced by
the contractor. These include:
* requirements traceability and walkthroughs,
* design walkthroughs,
* peer reviews,
* random inspections,
* formal software quality assurance audits conducted by independent
staff in the contractors' company.
The project team plans to assign technical monitors to monitor, assess,
document, and report on the contractor's performance. The team also
expects the contractor to perform its own quality control methods to
ensure that DRIS meets the government's requirements. The contractor's
quality assurance plan is due in June 2006.
However, the project team has not yet established a quality assurance
plan, assigned resources, or implemented a quality assurance process.
Specifically, the team has not yet determined how they will use the
planned inspections, reviews, and walkthroughs to evaluate product
quality. The delay in implementing this process is due in part to a
lack of Census policy requiring these processes be in place prior to or
soon after contract award. As a result, the project team is not able to
ensure that it will be able to identify and document any noncompliance
issues and take appropriate corrective actions.
Executive Oversight and Governance:
Office of Management and Budget and GAO guidance call for agencies to
establish IT investment management boards comprised of key executives
to regularly track the progress of major systems acquisitions. These
boards should have documented policies and procedures for management
oversight of IT projects and systems, and should be able to adequately
oversee the project's progress toward cost and schedule milestones and
their risks. The board should also employ early warning systems that
enable it to take corrective actions at the first sign of cost,
schedule, and performance slippages.
The DRIS project received executive level oversight during early
project planning:
* the project was presented to key executives from the Department of
Commerce IT Review Board and the Census Information Technology
Governing Board (these boards include executive level managers from all
bureau directorates, including the chief information officer and the
chief financial officer),
* the project team established an acquisition review team consisting of
key information technology, acquisition, legal and program managers
from the Census Bureau and the Department of Commerce to review and
approve acquisition documents,
* the project manager participated in meetings with Commerce and Census
Bureau executives, and
* the project team worked with senior acquisition officials to develop
acquisition planning and solicitation documents.
Executive-level oversight for DRIS is to continue as follows:
* The DRIS project manager is reporting progress directly to the chief
of the Decennial Systems and Contracts Management Office.
* The DRIS Program Office is reporting weekly on the status of the DRIS
project to Decennial Management Division officials.
* The DRIS Program Office is expected to participate in quarterly
reviews with the Commerce Information Technology Review Board, Census
Bureau managers, and the Associate Director for Decennial Census
Leadership Team.
* Commerce and Census executive-level IT governing boards expect to
review the DRIS project as part of its annual preparation for meeting
OMB requirements, under the 2010 Testing, Evaluation, and System Design
program.
However, in June 2005, we reported that the Census Bureau's executive
oversight of IT projects was incomplete. [NOTE 6] Specifically, we
noted that the bureau lacked:
* written procedures outlining the IT investment board's operations and
ensuring consistent investment management and decision making practices
and:
* written policies and procedures for monitoring the progress of its IT
projects and systems.
As a result, we reported that the bureau lacks assurance that
investment oversight and decision making is being performed in a
consistent and repeatable manner, and that consistent and appropriate
actions will be taken when cost, schedule, and performance expectations
are not met. We recommended that the bureau develop and implement these
policies and procedures. The bureau agreed to implement these
recommendations, and expects to do so by the end of July 2006.
Until these efforts are completed, the bureau cannot ensure that it is
providing effective and consistent oversight for the DRIS project.
NOTES:
[5] While there have not been any contract modifications to date, the
project team expects to modify the contract to reflect the revised
schedule.
[6] GAO-05-661.
DRIS:
Conclusions:
While the DRIS project team has initiated important steps in
establishing a sound and capable project management office, it has not
yet completed important activities it needs to adequately manage this
acquisition. Incomplete project plans, requirements, and risk
management activities increase the risks that this project will
encounter problems in meeting cost and schedule expectations. Given
that the contract was awarded in October and is currently under way, it
is critical that the DRIS project office expeditiously put in place key
elements of a sound acquisition management capability.
One of the root causes of the project's delay in completing key
management activities is that the Census Bureau lacks organizational
policies for managing major acquisitions. As a result, the success of
major acquisitions such as DRIS and FDCA are extremely dependent on the
knowledge, skills, and qualifications of the project teams. Without a
minimum set of required steps and processes, Census acquisitions are at
increased risk of being run in an ad hoc and chaotic manner-potentially
resulting in increased project costs, delayed schedules, and
performance shortfalls.
DRIS:
Recommendations:
To ensure that the bureau adequately manages the DRIS project, we
recommend that the project office take the following six actions:
* complete the DRIS project plan and obtain stakeholders' commitment to
the plan before initiating further development work,
* obtain validation and management and customer approval of DRIS
requirements,
* staff teams and perform planned requirements management activities,
* develop mitigation plans with milestones for key activities, and
regularly brief senior managers on important risks,
* establish performance measures and monitor results for contractor and
project office activities, and:
* implement a quality assurance process by establishing
responsibilities for assuring product quality, and defining how
inspections, reviews, and walkthroughs will be used to measure quality.
Further, to ensure that the bureau improves its ability to manage this
and other acquisitions, we recommend that the Director of the Census
Bureau immediately establish and enforce a system acquisition
management policy that incorporates best practices in system and
software acquisition management (including those highlighted in this
briefing).
FDCA:
Overview, Status, and Plans:
The Field Data Collection Automation (FDCA) program is expected to
provide automation support for field data collection operations for the
2010 Census. Specific requirements include:
* office automation for the twelve regional census centers, the Puerto
Rico area office, and more than 450 local census offices,
* the telecommunications infrastructure for headquarters, regional, and
local offices,
* mobile computing devices for field workers,
* integration with other 2010 Census systems (e.g., DRIS and
headquarters processing systems), and:
* development, deployment, technical support, de-installation, and
disposal services.
Automating field data collection activities is expected to help reduce
overall 2010 Census costs as well as improve data quality and
operational efficiency.
Mobile computing devices are a key technology component of FDCA. Census
plans to have field-based enumerators use nearly 500,000 mobile
computing devices to support the following decennial census functions:
* case management and automated payroll for enumerators,
* address canvassing (visiting households to update address lists and
collect global positioning system coordinates to update maps),
* non-response follow-up (visiting households to obtain information
from those that did not provide responses via mail, Internet, or
phone),
* assignment updating to avoid unnecessary follow-up on late responses,
* data transmission, and:
* conducting additional interviews at a sample of households in order
to help measure census undercounts or overcounts.
In fiscal year 2002, the bureau conducted market research on purchasing
mobile computing devices and then purchased and tested these devices
during the 2004 census test. After encountering technical problems and
realizing that they lacked resources and expertise to manage the
project in-house, bureau officials decided to use an integration
contractor to help develop and manage the FDCA project.
The bureau has:
* established a FDCA project office in January 2005,
* held an industry symposium in February 2005,
* issued a draft RFP and a pre-solicitation notice in April 2005,
requesting that vendors submit examples of previous experience with
projects similar to FDCA and describe the challenges facing the FDCA
project. (Based on these proposals, the bureau then invited five
vendors to develop and test prototypes for address canvassing. Of the
five, three vendors chose to do so),
* released the final RFP in June 2005, and:
* conducted a technical exchange period from October to December 2005,
during which the three vendors developed prototypes and the FDCA
project team evaluated draft proposals.
Project officials acknowledge that the FDCA schedule is ambitious. Key
near-term activities include:
Final proposals and prototypes due: January 20, 2006:
Final evaluation: Feb. 6-Mar. 17, 2006:
Source evaluation board review/decision: March 20-30, 2006:
Contract award: March 31, 2006:
Integrated baseline review: March 31-May 31, 2006:
Dress rehearsal offices open: January 2007 (FDCA office automation
ready):
Dress rehearsal address canvassing: April-May 2007 (using FDCA mobile
computing devices):
The FDCA contract is expected to be a cost-reimbursement contract with
multiple incentives. Cost estimates for FDCA are considered procurement
sensitive.
The FDCA RFP calls for a baseline planning period and three execution
periods as follows:
Baseline planning period: March 2006-May 2006:
* involves the project team and contractor reaching agreement on
schedule, cost, quality, scope, and technical performance measurement
for the first contract execution period.
Execution period 1: June 2006-December 2008:
* includes activities leading up to and during the 2008 dress
rehearsal:
Execution period 2: January 2009-September 2011:
* includes activities leading up to and during the 2010 Census:
Execution period 3: August 2010-December 2011:
* includes activities to wrap up operations after the completion of the
2010 Census:
FDCA:
Summary - Acquisition Management Capabilities:
Summary of the status of FDCA acquisition management capabilities.
[See PDF for image]
[End of table]
FDCA:
Acquisition Management Capabilities:
Project and acquisition planning:
Effective project planning involves establishing and maintaining plans
defining project scope and activities, including overall budget and
schedule, key deliverables, and milestones for key deliverables.
Project planning also involves ensuring that the project team has the
skills and knowledge needed to manage the project and obtaining
stakeholder commitment to the project plan. Effective acquisition
planning involves developing an acquisition strategy that includes
objectives, projected costs and schedules, and risks.
Regarding project and acquisition planning, the FDCA project office
has:
* defined the overall project scope and obtained an independent
government cost estimate,
* developed a project management plan that documents objectives and
high-level plans for managing the acquisition,
* established a project team with experience and certification in
project management;
* involved stakeholders in planning, and:
* developed a high-level acquisition strategy that includes objectives,
projected costs, and schedules.
However, the office has not yet:
* established an overall project schedule (including deliverables and
milestones) for the project team and for the contractor for each
contract execution period, or:
* obtained stakeholder commitment to a comprehensive project management
plan that includes the project's estimated costs, budget, and
schedules.
Bureau officials expect the project's costs and many management
processes to be finalized shortly after the contract is awarded in
early 2006, and they plan to work with the contractor to complete a
master product schedule and integrated baseline review during the
initial baseline planning period. By that time, Decennial Management
Division (DMD) officials also plan to have the schedule for key dress
rehearsal activities complete so that it can be considered during the
integrated baseline review.
In commenting on a draft of this briefing, bureau officials reiterated
that they have an independent government cost estimate, an overall
budget that they did not share with us due to its sensitivity, and an
overall schedule for census activities. Officials also noted that they
do not believe they can finalize cost, budget, deliverables, and
milestones until after the contract is awarded because they are
contracting for a solution-not a set of services or specific products.
While we acknowledge the various activities that the bureau has
undertaken to begin planning the FDCA project and the fact that a
project plan will continue to evolve as the program proceeds, the FDCA
project management plan does not provide stakeholders and other
reviewers a comprehensive overview of the project's estimates of costs,
schedules, and deliverables. Such a plan is often used to form a
baseline for the program and to obtain buy in from stakeholders.
Until the bureau completes key activities, including establishing
budgets and schedules and documenting stakeholder commitment to its
plans, it lacks assurance that reasonable plans for the project have
been developed, and faces increased risk that the project may not
achieve expected cost, schedule, and technical performance goals.
Solicitation:
This activity includes identifying the needs within a particular
acquisition, developing and implementing a solicitation plan, preparing
for the evaluation of responses, issuing a solicitation package,
evaluating responses, conducting supporting negotiations, and making
recommendations for award of the contract.
The project office has:
* identified needs for the FDCA acquisition,
* developed criteria for evaluating vendor proposals,
* generated the solicitation package,
* developed a source selection plan, and:
* conducted training on the evaluation process.
In addition, the project office has begun evaluating proposals, has
completed a technical exchange process, and plans to obtain
recommendations from the Source Selection Board for the award of the
contract in March 2006.
Requirements Development and Management:
Requirements development involves eliciting, analyzing, and validating
customer and stakeholder needs and expectations. Requirements
management involves establishing an agreed-upon set of requirements,
ensuring traceability between operational and product requirements, and
managing any changes to the requirements in collaboration with
stakeholders.
To develop and manage requirements,
* DMD developed draft operational requirements for the 2010 Decennial
Census,
* the project office and DMD developed FDCA-specific functional
requirements (listed in the RFP) with participation from stakeholders,
* the project office selected automated tools for tracking requirements
in the RFP and proposed changes to those requirements,
* the project office drafted a requirements management plan, and:
* DMD developed a detailed change control process for managing
requirements for all 2010 decennial systems, including FDCA.
Further, DMD plans to staff the implementation planning teams
responsible for managing changes to requirements, and initiate
requirements management activities after the FDCA contract is awarded.
However, DMD has not yet:
* validated and approved a baseline set of operational requirements,
or:
* ensured traceability between its operational requirements and the
FDCA RFP.
DMD officials attribute the delay in finalizing operational
requirements primarily to the challenge of managing the magnitude and
scope of the census and its unique environment. Until the bureau
finalizes its operational requirements for the census and ensures that
the FDCA RFP is consistent with the baseline requirements, the project
will be at risk of later changes to the requirements, potentially
affecting its ambitious development and implementation schedule. This
is especially important given the acknowledged programmatic risk that
there could be late changes in census scope.
Risk Management:
An effective risk management process identifies potential problems
before they occur, so that risk-handling activities may be planned and
invoked as needed across the life of the product and project in order
to mitigate adverse impacts on achieving objectives. Key activities
include identifying and analyzing risks, assigning resources,
developing risk mitigation plans and milestones for key mitigation
deliverables, briefing senior-level managers on high-priority risks,
and tracking risks to closure.
The project office has:
* developed a draft risk management process and:
* identified a number of high-level risks for the FDCA project,
including:
- an ambitious schedule;
- system design faults;
- late changes in project scope;
- GPS signal not available;
- maturity of FDCA contract management;
- continuity of funding.
To manage selected schedule and technical risks, the project office
has:
* adopted an approach calling for pre-award prototype development to
address widely acknowledged schedule and technical risks,
* incorporated an assessment of risk in its evaluation of vendor
proposals, evaluated vendors' approaches to risk management in their
responses to scenarios provided by the FDCA project team, and:
* briefed senior management on these risks.
However, the team has not yet implemented its risk management process.
Specifically, the project team has not yet:
* revisited or analyzed the identified risks,
* begun prioritizing and tracking project risks,
* assigned resources to manage risks, or:
* documented risk mitigation plans.
Bureau officials reiterated that they have managed FDCA risks
throughout the pre-award process and plan to continue to do so after
the contract is awarded, though not in the more formal manner outlined
in our study of best practices.
While we acknowledge that the program office continues to work to
address key risks, ad hoc risk management activities are not a
substitute for a more formal process where risks are routinely
identified, prioritized, and mitigated. Until the project team
implements an effective risk management process, it will lack a
mechanism to address known and unknown problems. Without such a
process, potential problems are more likely to become actual problems
and have adverse effects on objectives-including FDCA cost, schedule,
and performance.
Contract Tracking and Oversight/Project Monitoring and Control:
These processes provide oversight of the contractor's and the project
office's performance, in order to allow appropriate corrective actions
if actual performance deviates significantly from the plan. Key
activities in tracking both the contractor's and the project office's
performance include the selection of performance measures,
communicating status, taking corrective actions, and determining
progress.
The project team has:
* incorporated requirements in the RFP for earned value management and
other status reporting by the contractor,
* hired staff with contract surveillance experience,
* provided training on contract surveillance to selected project staff,
and:
* planned to oversee the contractor using checklists, reports, site
visits, vendor deficit tracking, and risk assessment.
However, the project team has not yet:
* developed detailed procedures and metrics for contract monitoring,
or:
* developed a full set of performance measures for internal project
office performance, or begun tracking this performance.
Project officials acknowledged the need for additional planning for
these processes and activities, but do not plan to fully implement many
aspects of these processes until after contract award. In commenting on
a draft of this briefing, bureau officials stated that there would be
little value in establishing metrics in advance of having a FDCA
solution and that they plan to work with the winning vendor to
establish meaningful metrics. Officials also stated that they have
already begun tracking internal office performance and noted that they
have already submitted earned value management metrics for the program
office to the Department of Commerce.
Given the importance of monitoring program office and contractor
performance, best practices show that it is not too soon to begin
identifying performance measures and developing a process for
monitoring and managing both the program office and contractor's
performance. Further, earned value management metrics are valuable, but
do not comprise a comprehensive set of metrics for monitoring a
program. The program office may identify and track other performance
measures, including metrics for changes in risks and requirements.
Moving forward, it will be important for the project team to establish
strong project monitoring and control over internal performance as well
as solid processes for tracking and overseeing the FDCA contractor's
progress. Until it does so, the project faces increased risk of delays
in identifying problems and taking appropriate corrective actions.
Process and Product Quality Assurance:
This process area provides staff and management with objective insight
into processes and associated work products. This includes the
objective evaluation of project processes and products against approved
descriptions and standards. Key activities include developing a quality
assurance plan, assigning resources to quality assurance activities,
and implementing quality assurance activities. Through quality
assurance, the project team is able to identify and document
noncompliance issues and provide appropriate feedback to project staff.
The project team has:
* drafted plans for quality assurance, contract surveillance, and award
fee determination that call for inspections, random sampling, periodic
surveillance, customer feedback, and monitoring,
* provided selected staff with training on quality assurance and
surveillance planning,
* begun planning to prepare staff for their roles in quality assurance,
* assigned a quality assurance manager, and:
* developed plans for a principal technical monitor and 12-15 technical
monitors to evaluate contractor work products and processes.
The project team plans to implement these quality assurance practices
after contract award.
Executive Oversight and Governance:
The Office of Management and Budget and GAO guidance call for agencies
to establish IT investment management boards comprised of key
executives to regularly track the progress of major systems
acquisitions. These boards should have documented policies and
procedures for management oversight of IT projects and systems, and
should be able to adequately oversee the project's progress toward cost
and schedule milestones and their risks. The board should also employ
early warning systems that enable it to take corrective actions at the
first sign of cost, schedule, and performance slippages.
The FDCA project received executive-level oversight during early
project planning.
* The project was presented to key executives from the Department of
Commerce's IT Review Board and the Census Information Technology
Governing Board.
* The project manager participated in meetings with Commerce and Census
Bureau executives.
* The project reported to a FDCA steering committee comprised of
executives from key bureau directorates.
* The project team worked closely with senior acquisition officials to
develop acquisition planning and solicitation documents.
Executive-level oversight of FDCA is expected to continue:
* the FDCA Program Office is expected to report weekly on the status of
the FDCA project to Decennial Management Division officials,
* the FDCA Program Office is expected to participate in quarterly
reviews with the Commerce Information Technology Review Board and
bureau managers, and
* Commerce and Census executive-level IT governing boards expect to
review the FDCA project as part of its annual budgetary planning, under
the 2010 Testing, Evaluation, and System Design program.
As previously noted, we reported in June 2005 on weaknesses in the
bureau's executive oversight of IT projects. Specifically, we reported
that, because the bureau did not have written procedures on how
executive oversight was to be performed, it had less assurance that
investment oversight and decision making was performed in a consistent
and reasonable manner. Without clear, documented, and consistent
governance procedures, the bureau cannot ensure that it is effectively
and consistently overseeing these investments.
FDCA:
Conclusions and Recommendations:
While the FDCA project team has initiated important steps in
establishing a capable project management office, it has not yet
completed important activities it needs to adequately manage this
acquisition. For example, the bureau has not yet implemented needed
processes for managing risks or measuring project performance. Without
these processes in place, the bureau remains at increased risk of not
developing and delivering FDCA on time and within budget. The project
may also be at risk of falling short of promised functionality. Given
the immovable deadline for performing the decennial census, the bureau
faces greater risk of cost overruns or limited system functionality.
To ensure that the bureau adequately manages the FDCA project, we
recommend that the FDCA program office take the following four actions:
* obtain stakeholder commitment to a project plan that includes
estimated project costs and schedules, including deliverables and
milestones,
* obtain validation and approval of baseline requirements,
* identify, prioritize, and assign responsibilities for risks, and
develop and implement risk mitigation plans and actions, and:
* develop internal and contractor performance measures and prepare to
track project cost, schedule, and performance.
Agency Comments and Our Evaluation:
In providing oral and email comments on a draft of this briefing,
Census Bureau officials, including the Associate Director for the
Decennial Census and the Assistant Director for Decennial Information
Technology and Geographic Systems, generally agreed with our
recommendations and stated that time constraints and budgets have
driven the bureau to proceed with its acquisitions before all of the
recommended activities have been completed. Officials stated that they
plan to complete these activities as soon as possible.
Bureau officials also stated that they intend to rely on the DRIS and
FDCA contractors to help refine requirements, project plans, and
performance measures. However, our experience in reviewing major system
acquisitions over the last several years has shown that it is important
for the government to exercise strong leadership in managing
requirements, plans, and measures.
Bureau officials also offered technical corrections which we have
incorporated as appropriate.
[End of section]
Appendix II: Objectives, Scope, and Methodology:
For both the Decennial Response Integration System and Field Data
Collection Automation program acquisitions, our objectives were to (1)
provide an overview, status, and plans (including costs, schedule, and
deliverables) and (2) determine if the bureau has capabilities in place
to successfully manage and oversee the acquisitions.
To provide each project's overview, status, and plans, we analyzed
current project documents, including plans, acquisition documents, and
deliverables; we also interviewed project officials.
To determine if the bureau had capabilities in place to successfully
manage and oversee the acquisitions, we identified sound IT systems
acquisition and management processes from industry standards, including
those developed by the Software Engineering Institute, and compared
them to the Census Bureau's practices for the selected acquisitions. We
evaluated the following processes:
* project and acquisition planning;
* solicitation;
* requirements development and management;
* risk management;
* contract tracking and oversight/project monitoring and control;
* process and product quality assurance; and:
* executive oversight and governance.
In each of the process areas listed above, we evaluated project
documentation and interviewed project officials to determine the status
of the bureau's efforts. We obtained comments from bureau officials,
including the Associate Director for the Decennial Census and the
Assistant Director for Decennial Information Technology and Geographic
Systems, on a draft of the briefing in attachment I. We conducted our
review between July 2005 and January 2006 at Census Bureau headquarters
in Suitland, Maryland, in accordance with generally accepted government
auditing standards.
(310847):
FOOTNOTES
[1] 13 U.S.C. 141 (a) and (b).
[2] GAO, Information Technology Management: Census Bureau Has
Implemented Many Key Practices, but Additional Actions Are Needed, GAO-
05-661 (Washington, D.C.: June 16, 2005).
[3] Earned value management is a project management tool that
integrates the investment scope of work with schedule and cost elements
for investment planning and control. This method compares the value of
work accomplished during a given period with that of the work expected
in the period. Differences in expectations are measured in both cost
and schedule variances.
[4] For more information see GAO, Decennial Census: Overview of
Historical Census Issues, GAO/GGD-98-103 (Washington, D.C.: May 1,
1998).
[5] An integrated baseline review is a joint assessment by the
contractor and the project team of the technical plans for a work
segment as well as the adequacy of the budgets, resources, and
schedules estimated to complete that work. This review results in a
detailed plan for work activities, costs, and schedules that is used as
the basis for tracking the earned value of the contractor's
deliverables.
[6] GAO, 2010 Census: Planning and Testing Activities Are Making
Progress, GAO-06-465T (Washington, D.C.: March 1, 2006).
[7] GAO, Internal Revenue Service: Assessment of Fiscal Year 2006
Budget Request and Interim Results of the 2005 Filing Season, GAO-05-
416T (Washington, D.C.: April 7, 2005); and GAO, Business Systems
Modernization: Internal Revenue Service's Fiscal Year 2005 Expenditure
Plan, GAO-05-774 (Washington, D.C.: July 22, 2005).
