Pacific Groundfish
Continued Efforts Needed to Improve Reliability of Stock Assessments Gao ID: GAO-04-606 June 4, 2004Because of concerns raised about the accuracy of National Marine Fisheries Service (NMFS) stock assessments, GAO reviewed the assessments for five species of Pacific groundfish: Pacific hake and four types of rockfish--bocaccio, canary, darkblotched, and yelloweye. Specifically, for these five species GAO (1) assessed the reliability of NMFS' stock assessments, (2) identified which relevant recommendations from NMFS' stock assessment improvement plan have been implemented and which have not, and (3) identified the costs associated with planned and ongoing improvements to groundfish stock assessments.
The reliability of the NMFS assessments is questionable for the five species GAO reviewed, although the assessments were based on the best information available at the time they were conducted. According to NMFS officials and a National Research Council report, to obtain reliable results each stock assessment should include at least one NMFS data source of sufficient scope and accuracy because such data are derived from unbiased, statistical designs. However, in the yelloweye assessment, no NMFS data were used, and in the darkblotched, canary, and bocaccio assessments, the NMFS data were limited because the NMFS' surveys were conducted in trawlable waters only. A 2003 NMFS report concluded that darkblotched groundfish are less abundant and bocaccio and canary are more abundant in untrawlable waters. Also for all five species, NMFS lacks a standard approach for ensuring the reliability of non-NMFS data used in stock assessments. Some assessors reviewed the quality of non-NMFS data; others did not. The assessors who reviewed the quality of the non-NMFS data found errors that made some of the data unusable or that could have impaired the reliability of certain stock assessments. Finally, for four species, the stock assessment reports were questionable because they did not present the uncertainty associated with the population estimates. For example, the canary stock assessment review panel recommended that standard estimates of uncertainty be included in the assessment report because without them it is difficult to determine their reliability. NMFS has taken steps to implement some of the recommendations contained in the NMFS stock assessment improvement plan, but much remains to be done. NMFS has concentrated its efforts mostly on improving data quantity. For example, NMFS increased the frequency of groundfish stock assessments and extended the geographic ranges of the shelf and slope surveys to cover over 300 more miles along the southern California coast. However, because of staffing and funding limitations, NMFS has not yet implemented many of the recommendations aimed at obtaining more types of data and improving data quality. For example, NMFS has not collected enough ecosystem data, and the frequency and range of recruitment surveys (estimated production of new members of a fish population) are limited. Finally, because of other program priorities, NMFS has not implemented the recommendation to create a comprehensive plan that combines the improvement plan and its complementary plans. NMFS records indicate at least $8.9 million is needed to complete ongoing and planned stock assessment improvements--$2.6 million that NMFS' Northwest Fisheries Science Center requested but did not receive in fiscal years 2001 to 2003, and $6.3 million requested for fiscal years 2004 and 2005. It will cost about (1) $7.7 million to improve the types of data used, such as more untrawlable water and recruitment surveys and (2) $1.2 million to improve the quality of data used in stock assessments, such as enhanced calibration of vessel equipment and standardized trawl survey procedures. The actual cost of the remaining improvements may be even higher than the $8.9 million estimated because the estimates primarily reflect the amount of money that agency officials believed could be realistically obtained, rather than what the improvements might cost.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-606, Pacific Groundfish: Continued Efforts Needed to Improve Reliability of Stock Assessments
This is the accessible text file for GAO report number GAO-04-606
entitled 'Pacific Groundfish: Continued Efforts Needed to Improve
Reliability of Stock Assessments' which was released on June 14, 2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao. gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
June 2004:
PACIFIC GROUNDFISH:
Continued Efforts Needed to Improve Reliability of Stock Assessments:
GAO-04-606:
GAO Highlights:
Highlights of GAO-04-606, a report to congressional requesters
Why GAO Did This Study:
Because of concerns raised about the accuracy of National Marine
Fisheries Service (NMFS) stock assessments, GAO reviewed the
assessments for five species of Pacific groundfish: Pacific hake and
four types of rockfish”bocaccio, canary, darkblotched, and yelloweye.
Specifically, for these five species GAO (1) assessed the reliability
of NMFS‘ stock assessments, (2) identified which relevant
recommendations from NMFS‘ stock assessment improvement plan have been
implemented and which have not, and (3) identified the costs associated
with planned and ongoing improvements to groundfish stock assessments.
What GAO Found:
The reliability of the NMFS assessments is questionable for the five
species GAO reviewed, although the assessments were based on the best
information available at the time they were conducted. According to
NMFS officials and a National Research Council report, to obtain
reliable results each stock assessment should include at least one
NMFS data source of sufficient scope and accuracy because such data
are derived from unbiased, statistical designs. However, in the
yelloweye assessment, no NMFS data were used, and in the darkblotched,
canary, and bocaccio assessments, the NMFS data were limited because
the NMFS‘ surveys were conducted in trawlable waters only. A 2003 NMFS
report concluded that darkblotched groundfish are less abundant and
bocaccio and canary are more abundant in untrawlable waters. Also for
all five species, NMFS lacks a standard approach for ensuring the
reliability of non-NMFS data used in stock assessments. Some assessors
reviewed the quality of non-NMFS data; others did not. The assessors
who reviewed the quality of the non-NMFS data found errors that made
some of the data unusable or that could have impaired the reliability
of certain stock assessments. Finally, for four species, the stock
assessment reports were questionable because they did not present the
uncertainty associated with the population estimates. For example, the
canary stock assessment review panel recommended that standard
estimates of uncertainty be included in the assessment report because
without them it is difficult to determine their reliability.
NMFS has taken steps to implement some of the recommendations contained
in the NMFS stock assessment improvement plan, but much remains to be
done. NMFS has concentrated its efforts mostly on improving data
quantity. For example, NMFS increased the frequency of groundfish stock
assessments and extended the geographic ranges of the shelf and slope
surveys to cover over 300 more miles along the southern California
coast. However, because of staffing and funding limitations, NMFS has
not yet implemented many of the recommendations aimed at obtaining more
types of data and improving data quality. For example, NMFS has not
collected enough ecosystem data, and the frequency and range of
recruitment surveys (estimated production of new members of a fish
population) are limited. Finally, because of other program priorities,
NMFS has not implemented the recommendation to create a comprehensive
plan that combines the improvement plan and its complementary plans.
NMFS records indicate at least $8.9 million is needed to complete
ongoing and planned stock assessment improvements”$2.6 million that
NMFS‘ Northwest Fisheries Science Center requested but did not receive
in fiscal years 2001 to 2003, and $6.3 million requested for fiscal
years 2004 and 2005. It will cost about (1) $7.7 million to improve the
types of data used, such as more untrawlable water and recruitment
surveys and (2) $1.2 million to improve the quality of data used in
stock assessments, such as enhanced calibration of vessel equipment and
standardized trawl survey procedures. The actual cost of the remaining
improvements may be even higher than the $8.9 million estimated because
the estimates primarily reflect the amount of money that agency
officials believed could be realistically obtained, rather than what
the improvements might cost.
What GAO Recommends:
GAO recommends that the Secretary of Commerce require the Director of
NMFS to take actions to improve the reliability of stock assessments,
such as continuing efforts to improve the quality and types of data
used in groundfish assessments, establishing a standard approach that
requires that data used in stock assessments be evaluated for
reliability, and requiring stock assessment reports clearly present the
uncertainties in the assessments.
NOAA generally agreed with the report‘s accuracy and agreed with the
report‘s recommendations, but expressed concern that the report‘s
conclusion could be misconstrued to infer that the assessments are
unreliable for use in managing the west coast groundfish fishery.
www.gao.gov/cgi-bin/getrpt?GAO-04-606.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Anu K. Mittal at (202)
512-3841 or MittalA@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Reliability of NMFS' Pacific Groundfish Stock Assessments Is
Questionable:
Some Recommended Stock Improvements Have Been Implemented, but Much
Remains to be Done:
Remaining Improvements Estimated to Cost at Least $8.9 Million:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Comments From the Department of Commerce:
GAO Comments:
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Staff Acknowledgments:
Tables:
Table 1: Stock Assessment Reliability Issues for Five Pacific
Groundfish Species:
Table 2: Pacific Groundfish Stock Assessment Improvement Funds
Requested and Received by Data Type and Quality, Fiscal Years 2001-
2005:
Figures:
Figure 1: Photograph of a Pacific Hake:
Figure 2: Photograph of a Bocaccio Rockfish:
Figure 3: Photograph of a Canary Rockfish:
Figure 4: Photograph of a Darkblotched Rockfish:
Figure 5: Photograph of a Yelloweye Rockfish:
Abbreviations:
NMFS : National Marine Fisheries Service:
NOAA: National Oceanic and Atmospheric Administration:
Letter June 4, 2004:
The Honorable Gordon H. Smith:
United States Senate:
The Honorable Ron Wyden:
United States Senate:
Pacific groundfish, 82 species of bottom-dwelling fish, such as several
species of rockfish and sole, have contributed to the economies of
fishing communities in California, Oregon, and Washington, generating
over $220 million in income in 2001. However, some populations of
Pacific groundfish have declined sharply because of natural conditions,
such as climate, as well as man-made conditions, such as overfishing.
In an effort to achieve sustainable populations, the National Marine
Fisheries Service (NMFS) significantly reduced the allowable harvest
for nine species of Pacific groundfish. NMFS made this decision based
on its stock assessments--which include estimates of species' total
biomass (weight of the total population)--and recommendations from the
Pacific Fishery Management Council, one of eight councils responsible
for managing fisheries in federal waters.
NMFS generally uses internal scientists or contracts with outside
experts to conduct stock assessments. These assessors use multiple
types of data, including data from external sources, such as state-
collected data on the poundage and species of fish brought ashore by
commercial and recreational fishermen. NMFS' Northwest Fisheries
Science Center (Northwest Center), which coordinates Pacific groundfish
stock assessments, also collects its own data through fish population
surveys, particularly two key surveys: (1) the shelf and slope bottom
trawl survey, in which a trawl net is dragged along the ocean floor,
and the resulting catch is sorted, counted, and biologically examined
(e.g., age and sex) and (2) the acoustic survey, in which sound waves
are used to measure density of schools of groundfish in mid-level
waters; bottom trawls are not feasible in these waters. Fish survey and
other data are entered into computerized mathematical models (stock
assessment models), which estimate the total biomass for the fish
species being assessed. However, concerns have been raised about the
accuracy of stock assessments and therefore whether they can be relied
upon to specify the amount of fish that can be harvested to ensure a
sustainable population.
In 1999, NMFS commissioned a national task force to review the stock
assessment process across the agency. Task force members were from NMFS
headquarters and five of the six NMFS science centers, including the
Northwest Center. In the October 2001 Marine Fisheries Stock Assessment
Improvement Plan,[Footnote 1] the task force identified a lack of
adequate data as the greatest impediment to producing reliable stock
assessments and recommended actions to improve the quantity, quality,
and types of data used in the models that estimate biomass. The task
force also identified a need to better communicate and quantify
uncertainties in these models.
In this context, you requested that we review NMFS' stock assessments
for five species of Pacific groundfish: Pacific hake as well as four
types of rockfish--bocaccio, canary, darkblotched, and yelloweye.
Specifically, for these five species you asked us to (1) assess the
reliability of NMFS' stock assessments, (2) identify which relevant
recommendations from the stock assessment improvement plan have been
implemented and which have not, and (3) identify the estimated costs
associated with planned and ongoing improvements to groundfish stock
assessments.
To address these issues, we reviewed key laws and stock assessment
studies (2002 and 2004 studies for Pacific hake, 2002 and 2003 studies
for bocaccio, 2002 study for canary, 2000 and 2003 studies for
darkblotched, and 2001 and 2002 studies for yelloweye), along with the
types of fish population surveys used and the controls over stock
assessment data. We spoke with officials from NMFS, the Oregon
Department of Fish and Wildlife, the Washington Department of Fish and
Wildlife, the California Department of Fish and Game, the Pacific
Fishery Management Council, the Pacific States Marine Fisheries
Commission, environmental groups, and industry associations, as well as
fishermen and academics. We also reviewed NMFS' budget documents to
develop cost estimates. We did not simulate NMFS' stock assessment
models or evaluate the mathematical and statistical methodologies used
in the models. We conducted our review from May 2003 through April 2004
in accordance with generally accepted government auditing standards.
See appendix I for additional details on our scope and methodology.
Results in Brief:
For a variety of reasons, the reliability of the NMFS assessments is
questionable for the five species we reviewed, although the assessments
were based on the best information available at the time they were
conducted. First, according to NMFS officials and a 1998 National
Research Council report, to obtain reliable results, each stock
assessment should include at least one source of NMFS-collected data of
sufficient scope and accuracy because such data are derived from
unbiased, statistical, and scientific designs. However, the four
rockfish assessments did not meet this standard, although the
assessment for Pacific hake did. NMFS did not use any of its own data
in the yelloweye assessment, and the NMFS data used in the bocaccio,
canary, and darkblotched assessments were limited because NMFS
conducted its surveys in trawlable waters only; these fish also inhabit
both trawlable and untrawlable waters. In the absence of data from
rocky, untrawlable waters, the assessors estimated overall biomass
using the NMFS data collected from the trawlable areas. However, in
2003, NMFS found that darkblotched are less abundant and bocaccio and
canary are more abundant in untrawlable waters. As a result, some
rockfish populations may be overstated, while others may be
understated. Second, for all the species, NMFS lacks a standard
approach for ensuring the reliability of non-NMFS data used in the
stock assessments. As a result, some assessors reviewed the quality of
these data while others did not. The assessors who reviewed the quality
of the non-NMFS data found errors that they believed made the data
unusable or impaired the reliability of the stock assessment. Finally,
the reliability of four of the stock assessment reports--bocaccio,
canary, darkblotched, and yelloweye--is questionable because they did
not present the uncertainty associated with the population estimates.
The National Research Council, NMFS, and GAO agree that estimates of
uncertainty should be included in stock assessment reports because
without them it is difficult to determine their reliability.
NMFS has taken steps to implement the NMFS stock assessment improvement
plan recommendations to improve stock assessments, but much remains to
be done to enhance their reliability. While the task force considered
the lack of adequate data--in quantity, quality, and type--as the
greatest impediment to accurate stock assessments, the Northwest Center
has concentrated most of its efforts on improving data quantity. For
example, starting in 2003, the Northwest Center extended the geographic
ranges of the shelf and slope surveys, expanding them to cover over 300
additional miles along the southern California coast. It also increased
the frequency of the groundfish shelf and slope survey from triennially
to annually and the frequency of the Pacific hake acoustic survey from
triennially to biennially. However, NMFS has not yet fully implemented
many of the recommendations aimed at obtaining more types of data and
improving data quality. For example, although Northwest Center
officials recognize that recruitment surveys (estimated production of
new members of a fish population) should be conducted biannually along
the entire coast, these surveys are currently conducted only annually
and only in selected regions. While Northwest Center officials also
recognize that actions, such as calibrating vessel equipment to ensure
comparable survey data, are needed to improve data quality, these
actions have not yet been fully implemented. Finally, NMFS has not
implemented the recommendation to create a comprehensive plan that
combines the improvement plan and related plans to develop integrated
program initiatives. Although NMFS officials recognize a need for such
a plan, other priorities have precluded them from developing it.
According to NMFS funding and budget documents, NMFS needs at least
$8.9 million to complete ongoing and planned improvements to its stock
assessments of Pacific groundfish--$2.6 million that the Northwest
Center had requested but did not receive in fiscal years 2001 through
2003, and $6.3 million requested for fiscal years 2004 and 2005.
Specifically, according to these records, the Northwest Center has the
following funding needs:
* $7.7 million to improve the types of data used, including $2.4
million for surveys of untrawlable waters, $2.1 million to expand
acoustic and recruitment surveys, and $3.2 million to collect ecosystem
data and:
* $1.2 million to improve the quality of data used in stock
assessments, including $600,000 to enhance the calibration of vessel
equipment, $525,000 to develop and implement methods to more accurately
distinguish among groundfish species, and $75,000 to standardize trawl
survey procedures.
The actual cost of implementing improvements to Pacific groundfish
stock assessments may be even higher than the $8.9 million estimated
because the Northwest Center's budget requests primarily reflect the
amount of money the Center believed it could realistically obtain,
rather than what the improvements would cost, according to NMFS
officials.
Background:
The Magnuson-Stevens Fisheries Conservation and Management Act granted
responsibility for managing marine resources to the Secretary of
Commerce. [Footnote 2] The Secretary delegated this responsibility to
NMFS, which is part of Commerce's National Oceanic and Atmospheric
Administration (NOAA). The act also established eight regional fishery
management councils, each responsible for making recommendations to the
Secretary of Commerce about managing fisheries in federal waters. The
eight fishery management councils--consisting of fishing industry
participants, state and federal fishery managers, and other interested
parties--and their areas of responsibility include the following:
* Caribbean Council, covering waters off the U.S. Virgin Islands and
the Commonwealth of Puerto Rico;
* Gulf of Mexico Council, covering waters off Texas, Louisiana,
Mississippi, Alabama, and the west coast of Florida;
* Mid-Atlantic Council, covering waters off New York, New Jersey,
Delaware, Maryland, Virginia, and North Carolina;
* New England Council, covering waters off Maine, New Hampshire,
Massachusetts, Rhode Island, and Connecticut;
* North Pacific Council, covering waters off Alaska;
* Pacific Council, covering waters off California, Oregon, and
Washington;
* South Atlantic Council, covering waters off North Carolina, South
Carolina, Georgia, and the east coast of Florida; and:
* Western Pacific Council, covering waters off Hawaii, American Samoa,
Guam, the Commonwealth of the Northern Mariana Islands, and uninhabited
U.S. territories in the Western Pacific.
In addition to these eight councils, NMFS has six regional science
centers, which are responsible for generating the scientific
information necessary for the conservation, management, and use of each
region's marine resources. The six fishery science centers and their
areas of responsibility are as follows:
* Alaska Center, covering the coastal oceans off Alaska and parts of
the west coast of the United States;
* Northeast Center, covering waters along the Northeast Continental
Shelf from the Gulf of Maine to Cape Hatteras, North Carolina;
* Northwest Center, covering the northeast Pacific Ocean, primarily
waters off the coasts of California, Oregon, Washington, and British
Columbia;[Footnote 3]
* Pacific Islands Center, covering the central and western Pacific
Ocean;
* Southwest Center, primarily covering waters off the coast of
California and areas throughout the Pacific and Antarctic
Oceans;[Footnote 4] and
* Southeast Center, covering waters along the continental southeastern
United States as well as Puerto Rico and the U.S. Virgin Islands.
The Magnuson-Stevens Act, as amended by the Sustainable Fisheries
Act,[Footnote 5] also established national standards for fishery
conservation and management. These standards deal with preventing
overfishing, using scientific information, using fishery resources
efficiently, minimizing bycatch,[Footnote 6] and minimizing
administrative costs. The regional councils use these standards to
develop appropriate plans for conserving and managing fisheries under
their jurisdiction, including measures to prevent overfishing and to
rebuild overfished stocks as well as measures to protect, restore, and
promote the long-term health and stability of the fishery.
In 1982, the Pacific Fishery Management Council (Pacific Council)
released its initial Fishery Management Plan for groundfish. The
Pacific Council's goal is to have long-range plans for managing
groundfish fisheries that will promote a stable planning environment
for the seafood industry, including marine recreation interests, and
will maintain the health of the resource and environment. To help
achieve these goals, stock assessments are conducted on groundfish
species to estimate fish populations. Since 1995, the Northwest Center
has had lead responsibility for conducting stock assessments on Pacific
groundfish. The Northwest Center receives assistance from other NMFS
science centers, such as the Southwest Center, which conducted the
bocaccio assessment.
Stock assessments are the biological evaluation of the status of fish
stocks. Stock assessments provide official estimates in key areas, such
as the size of the stock population, the size of the spawning
population, the amount of fish that have died (fish mortality), and the
estimated number of fish at a particular young age (recruitment). Stock
assessments form the scientific basis used by regional councils to
determine biologically sustainable harvests and guide the monitoring
and rebuilding of overfished and threatened stocks. For example,
regional councils use stock assessments and other indicators of
biological productivity to recommend to NMFS a maximum, or total
allowable catch, in a particular fishery--typically for a year. Stock
assessments are a key tool for managing fisheries. Without stock
assessments, fishery managers would have limited information about the
status of fisheries in making decisions about setting harvest levels
and developing plans to rebuild overfished stocks.
For each species, the assessor reviews previous stock assessments,
gathers available data about the species being assessed, runs the data
through computer-generated models, and estimates the species' total
biomass. Stock assessors use NMFS-collected data, such as stock surveys
conducted on NOAA vessels or contracted commercial fishing vessels, as
well as data collected by non-NMFS sources, such as commercial and
recreational catch data collected by state agencies. The following six
key types of data are collected:
* Stock abundance--surveys of how many fish constitute a stock's total
size or weight.
* Commercial and recreational fisheries data--the amount and
composition of fish caught from a particular stock, whether caught
intentionally by commercial and recreational fishermen or
unintentionally caught and discarded. Data sources include fishing
logbooks, dockside samples, and onboard observations, among others.
* Life history--biological data, such as the age and sex composition of
the stock, age at first maturity, fertility, average lifespan, and
natural mortality.
* Ecosystem relationships--data on the relationship between a fish
stock and its physical environment, as well the relationship of a fish
stock to other species.
* Recruitment research--data on the abundance of juvenile and larval
fish (fish at their earliest stage), which helps scientists forecast
the size of a particular stock in the future.
* Synoptic oceanographic sampling--data on the ocean ecosystem, such as
water temperature or salinity, plankton composition, or ocean currents.
For each stock assessment, a review panel, consisting of NMFS
scientists and outside experts, independently reviews the methodology
of the assessment and works with the assessor to ensure their comments
are adequately addressed.
Through 2003, 24 of the 82 species of Pacific groundfish have had a
full quantitative stock assessment. Relying on these assessments, NMFS
has declared that nine species of Pacific groundfish are overfished,
including the five species of Pacific groundfish we reviewed in this
report--Pacific hake as well as bocaccio rockfish, canary rockfish,
darkblotched rockfish, and yelloweye rockfish. Rockfish are long-lived,
late-maturing and slow-growing species, making them particularly
susceptible to overfishing. More specifically Pacific hake, also called
Pacific whiting, is generally found off the west coast of North
America. It is one of many species of hake distributed in the Atlantic
and Pacific oceans. Fishing of Pacific hake primarily takes place off
the coasts of northern California, Oregon, Washington, and British
Columbia. Fishermen use mid-water trawls and generally fish over the
ocean bottom at depths of 100 to 500 meters. Pacific hake was declared
overfished in 2002 because the 2002 stock assessment estimated Pacific
hake biomass at 700,000 metric tons. By 2004, the biomass was
estimated at between 2.7 and 4.2 million metric tons, and Pacific
hake is no longer considered overfished. Figure 1 shows a picture of a
Pacific hake.
Figure 1: Photograph of a Pacific Hake:
[See PDF for image]
[End of figure]
Bocaccio rockfish generally inhabit waters off the coast of northern
Baja, Mexico to Alaska. Bocaccio, commonly sold at market as "red
snapper," are commercially fished using trawls, hook-and-line and
gillnets. Adult bocaccio commonly live over rocky areas or open areas
of the ocean's floor to about 320 meters. Bocaccio were formally
declared to be "overfished" in 1999. The 2003 stock assessment
estimated the bocaccio biomass at 7,133 metric tons. Figure 2 shows a
picture of a bocaccio rockfish.
Figure 2: Photograph of a Bocaccio Rockfish:
[See PDF for image]
[End of figure]
Canary rockfish inhabit the northeastern Pacific Ocean, from northern
Baja, Mexico to the western Gulf of Alaska. Adult canary rockfish are
primarily found along the continental shelf, from 46 to 457 meters
deep. Canary rockfish are harvested commercially using trawl nets and
hook-and-line and are also considered an important species for
recreational fishermen. NMFS declared canary rockfish as overfished in
2000. The 2002 stock assessment estimated the canary rockfish biomass
at 6,197 metric tons. Figure 3 shows a picture of a canary rockfish.
Figure 3: Photograph of a Canary Rockfish:
[See PDF for image]
[End of figure]
Darkblotched rockfish are found in the waters from Santa Catalina
Island, California to the Bering Sea on soft bottom areas at about 29
to 549 meters deep. Commercial fishery concentrations are located off
the coasts of California and Oregon. Darkblotched rockfish are caught
primarily by commercial trawls and contribute to both commercial and
recreational fishing. NMFS determined that darkblotched rockfish was
overfished in 2000, when the last full stock assessment was conducted;
it was updated in 2003. This update estimated the darkblotched rockfish
biomass at 7,266 metric tons in 2001. [Footnote 7] Figure 4 shows a
picture of a darkblotched rockfish.
Figure 4: Photograph of a Darkblotched Rockfish:
[See PDF for image]
[End of figure]
Yelloweye rockfish almost exclusively inhabit rocky areas from northern
Baja, Mexico to the Aleutian Islands, Alaska. Yelloweye rockfish, found
in depths ranging from 15 to 550 meters, are caught using both trawl
nets and line gear and are highly prized by both commercial and
recreational fishermen. Stock assessments for yelloweye rockfish were
first conducted in 2001, and NMFS determined that the species was
overfished in 2002. The 2002 stock assessment estimated the yelloweye
biomass at 2,325 metric tons in 2001. [Footnote 8] Figure 5 shows a
picture of a yelloweye rockfish.
Figure 5: Photograph of a Yelloweye Rockfish:
[See PDF for image]
[End of figure]
Reliability of NMFS' Pacific Groundfish Stock Assessments Is
Questionable:
The reliability of NMFS' stock assessments is questionable for the
Pacific hake and four rockfish species we reviewed, although they were
based on the best information available at the time the assessments
were conducted. The reliability of the stock assessments we reviewed is
questionable because (1) four of the assessments did not have at least
one NMFS-collected data source of sufficient scope and accuracy; (2)
NMFS lacked a standard process for assessing the reliability of non-
NMFS data used in all five assessments; and (3) for four of the
assessments, the stock assessment reports did not adequately identify
the uncertainty of the biomass estimates. (See table 1. ) To address
these limitations, the Northwest Center plans to increase the scope and
accuracy of its collected data, as additional funds become available;
is implementing changes that will help ensure the reliability of non-
NMFS data; and plans to update the stock assessment model to provide
uncertainty ranges for the 2005 stock assessments.
Table 1: Stock Assessment Reliability Issues for Five Pacific
Groundfish Species:
Species: Pacific hake;
NMFS-collected data available? Yes;
Standard data reliability testing conducted by NMFS? No;
Uncertainty ranges for biomass estimates provided? Yes.
Species: Bocaccio;
NMFS-collected data available? Limited[A];
Standard data reliability testing conducted by NMFS? No;
Uncertainty ranges for biomass estimates provided? No.
Species: Canary;
NMFS-collected data available? Limited[A];
Standard data reliability testing conducted by NMFS? No;
Uncertainty ranges for biomass estimates provided? No.
Species: Darkblotched;
NMFS-collected data available? Limited[A];
Standard data reliability testing conducted by NMFS? No;
Uncertainty ranges for biomass estimates provided? No.
Species: Yelloweye;
NMFS-collected data available? No;
Standard data reliability testing conducted by NMFS? No;
Uncertainty ranges for biomass estimates provided? No.
Source: GAO analysis of information provided by NMFS.
[A] NMFS-collected data are available for only the trawlable portion of
the species' habitat.
[End of table]
Absence of NMFS Survey Data Adversely Affects the Assessments'
Reliability:
Stock assessors use a variety of data, including NMFS data and non-NMFS
data, in developing their assessments. [Footnote 9] Two key pieces of
NMFS survey data are the shelf and slope bottom trawl survey and the
acoustic survey. Other NMFS data that assessors sometimes use include
larval surveys (data for fish in their earliest stage) and recruitment
surveys. The non-NMFS data assessors use include commercial catch data
and recreational catch data.
A 2002 National Research Council report found that the inclusion of
NMFS survey data was the best option for a reliable estimate of
abundance because such surveys use an unbiased statistical design,
control sampling locations, and provide for quality assurance. [Footnote
10] According to Northwest Center officials, to obtain reliable
results, each stock assessment should include at least one source of
NMFS-collected data of sufficient scope and accuracy because such
surveys are unbiased and scientifically designed.
Northwest Center officials raised concerns about basing assessments
solely on non-NMFS data such as commercial and recreational catch data.
Catch data do not provide the species' relative or absolute biomass,
according to NMFS officials. For example, catch data alone is
insufficient because:
* fishermen are not randomly sampling the ocean but are fishing areas
that they are allowed to fish and they believe have the most fish;
* fishing restrictions, such as a total allowable catch, can limit the
amount of fish being caught; and
* catch data have often been inaccurate for a variety of reasons, such
as imprecise accounting for dead fish tossed back into the ocean.
Although the assessors used several different data sources, four of the
five assessments did not use NMFS survey data or the NMFS data used
covered only a portion of the species' habitats. In the yelloweye
assessment, no NMFS survey data were available because yelloweye live
almost exclusively in the rocky habitat that NMFS trawl surveys cannot
cover. As a result, the yelloweye assessment was based solely on non-
NMFS data. Similarly, the NMFS survey data used in the bocaccio,
canary, and darkblotched assessments were limited in scope because the
surveys were conducted only in trawlable waters. Bocaccio, canary, and
darkblotched live in both the trawlable and untrawlable habitats. Using
data from trawl surveys conducted from 1977 through 1998, NMFS reported
in 2003 that 77 percent of the survey area was trawlable and 23 percent
was untrawlable. [Footnote 11] Lacking data on species in the 23 percent
of the ocean floor that is untrawlable, the assessors estimated the
overall biomass using the NMFS data collected from the trawlable areas.
However, the abundance in the trawlable area is not representative of
the abundance in the untrawlable area. The 2003 NMFS report also found
that darkblotched groundfish are less abundant in untrawlable waters,
while canary and bocaccio species are more abundant in untrawlable
waters. As a result, some rockfish populations may be understated while
others may be overstated. According to stock assessors, relying solely
on survey data from trawlable waters increases the uncertainty of stock
assessments.
In contrast, the fifth groundfish species, Pacific hake, lives
primarily in mid-water habitat; and so the concerns about the lack of
NMFS data in rocky, untrawlable habitats are not applicable.
NMFS Lacks a Standard Process for Ensuring the Reliability of Non-NMFS
Data Used in the Stock Assessments:
NMFS does not have a standard process for evaluating whether the non-
NMFS data used in its stock assessments are reliable. We believe that
certain internal control activities, such as a standard process for
ensuring data reliability, can help ensure that information used to
make management decisions is complete and accurate. [Footnote 12]
Lacking a standard process, some assessors reviewed the quality of the
raw non-NMFS data, while others did not. Assessors who reviewed for
data quality found mistakes that they believed made some of the data
unusable or that could have impaired the accuracy of the stock
assessments. For example, the assessor for the 2002 yelloweye stock
assessment found numerous errors in the recreational catch data, such
as attributing the catch from an entire fishing vessel to a single
fisherman, and thus did not use the data because doing so could have
resulted in overestimating the biomass. According to another stock
assessor, commercial catch data frequently have inconsistencies.
Specifically, the assessor said California, Oregon, and Washington
require fishermen to enter catch and location information into
logbooks, but logbooks are often incomplete and inaccurate. While the
stock assessment review panels evaluated the assessments, the panels
did not evaluate the quality of the raw data used in the assessments.
According to a Northwest Center official, several assessors have raised
concerns about data quality and accessibility in feedback meetings. In
response to these concerns, the Northwest Center official has recently
begun assigning data stewards to each data set used in its assessments.
Data stewards are responsible for helping assessors compile relevant
data and for conducting quality assurance and quality control checks on
the data. The Northwest Center plans to conduct a data quality workshop
in July 2004 to formally establish the roles and responsibilities of
the data stewards, with the intent of standardizing the data evaluation
process.
Stock Assessments for Four Species Did Not Provide Uncertainty
Estimates:
In 1998, the National Research Council recommended that NMFS include
realistic measures of uncertainty in its stock assessments. [Footnote
13] NMFS' 2001 stock assessment improvement plan also recognized the
need to better quantify and communicate the uncertainty in assessments.
In a review of the 2002 canary assessment, the stock assessment review
panel recommended that standard estimates of uncertainty be included in
future assessments because it is difficult to determine the reliability
of the stock assessment without them. [Footnote 14] Similarly, we
believe that estimates based on samples should have a range of
uncertainty to show the amount of variability in the
estimates. [Footnote 15] However, the bocaccio, canary, darkblotched,
and yelloweye assessments did not present a measure of uncertainty
associated with the biomass estimates.
Without uncertainty ranges, it is difficult for regional councils and
NMFS to know how much confidence they can have in relying on the
estimates for determining stock abundance and hence for setting
allowable harvests of the fish. For example, lacking uncertainty
ranges, the 2002 bocaccio stock assessment estimated a bocaccio biomass
of 2,914 metric tons in 2002. The 2003 assessment of bocaccio biomass,
however, estimated 6,506 metric tons in 2002--more than doubling the
previous estimate because of additional and updated data. With such
wide variations, it is important to provide uncertainty ranges,
otherwise management may make inappropriate decisions.
While assessors told us that their stock assessments included some
information about differences in estimated biomass when using different
data sources (sensitivity analyses), the mathematical model that NMFS
uses to estimate biomass (Stock Synthesis model) does not calculate
uncertainty ranges. NMFS officials told us that NMFS is updating the
model so that it can compute uncertainty ranges; NMFS expects to use
the updated model for all 2005 stock assessments. The Pacific hake
assessor used a mathematical model (AD Model Builder) that could
compute uncertainty ranges and included these ranges in the Pacific
hake assessment report.
Some Recommended Stock Improvements Have Been Implemented, but Much
Remains to be Done:
NMFS has taken some steps recommended in the Marine Fisheries Stock
Assessment Improvement Plan to improve the quantity, quality, and type
of data used in Pacific groundfish stock assessments, but much remains
to be done to make the assessments more reliable. The Northwest Center
has concentrated most of its efforts on implementing recommendations
aimed at obtaining more data. Recommendations aimed at increasing the
types of data and improving their quality have not yet been fully
implemented for a variety of reasons, such as staffing and funding
limitations. In addition, other program priorities have precluded NMFS
from implementing the recommendation to create a comprehensive plan
that incorporates the improvement plan and related plans so that it can
develop integrated program initiatives to improve stock assessments.
The Northwest Center Has Taken Some Steps Recommended for Improving
Stock Assessments:
The October 2001 stock assessment improvement plan identified three
scenarios (tiers) to consider when analyzing the resources needed to
improve stock assessments. The three tiers of assessment improvements
are as follows:
* Tier 1--improve stock assessments using existing data without
initiating new data collection programs.
* Tier 2--conduct baseline monitoring of species, which in most cases
requires sampling the species at least every 1 to 3 years, and
preferably at least every 1 to 2 years.
* Tier 3--implement "next generation" stock assessments by explicitly
incorporating ecosystem considerations, such as multispecies
interactions and environmental effects in assessments.
The improvement plan also made a number of recommendations to improve
stock assessments. The recommendations fall into the following four
categories:
* Data collection--pursue new initiatives to expand data collection
efforts that at a minimum bring stock assessment science to Tier 2. In
addition, continue to develop partnerships and cooperative research
programs with other entities, such as state agencies, commercial and
recreational fishing organizations, and individuals to improve the
quantity, quality, and type of data collected.
* Communication--educate constituents about NMFS' strategies for
improving stock assessments.
* Training--implement comprehensive training and staff development
programs for NMFS' analytical and quantitative staff, and augment
existing programs that support graduate students interested in stock
assessment science.
* Planning--develop integrated program initiatives by preparing a
comprehensive plan that combines the improvement plan with its
complementary plans.
Improvement in the quantity of data collected for use in stocks
assessments is a key component to achieving Tier 2 status. The
improvement plan states that as the quantity of the data increases, the
assessments become more reliable because the data cover a longer period
of time, producing better population trend information. Northwest
Center officials said that the quality of the data improves with more
frequent surveys and more randomly selected survey locations that, over
the long term, provide a better understanding of the variability
inherent in the population distribution and abundance. A better sense
of trends and variability allow for improved short-term predictions of
the status of the species.
The Northwest Center has concentrated most of its efforts on
implementing improvements in data quantity, such as more frequent
acoustic, and shelf and slope bottom trawl surveys. The following
illustrate some of the actions the Northwest Center took in 2003 to
improve data quantity: [Footnote 16]
* Increased the frequency of the Pacific hake acoustic survey from
triennially to biennially. Beginning in 2003, the survey was
restructured into a single, integrated survey with Northwest Center and
Canadian officials jointly planning all survey elements. Officials from
the Northwest Center and Canada now jointly conduct all of the acoustic
surveys.
* Increased the frequency of the groundfish shelf and slope bottom
trawl survey from triennially to annually, leveraging available
resources by cooperatively working with the fishing industry.
Specifically, contracting with private commercial fishing vessels to
conduct the surveys. According to Northwest Center officials, working
collaboratively with the fishing industry has afforded fishermen the
opportunity to become stakeholders in the data collection process.
* Extended the geographic range of the groundfish shelf and slope
bottom trawl survey. The surveys are now coastwide from Cape Flattery,
Washington to the Mexican border, adding over 300 more miles along the
southern California coast. Previous surveys ended at Morro Bay,
California.
Efforts continue to communicate the strategies needed to improve stock
assessments and to augment existing programs aimed at developing future
stock assessment scientists. For example, through the Pacific Fishery
Management Council, Northwest Center staff meet with their
constituents, such as representatives from state agencies and the
fishing industry, to discuss groundfish management issues. In addition,
the Northwest Center organized a series of public meetings to discuss
new initiatives that affect the stock assessment program. For example,
the Northwest Center held public meetings in several communities along
the Pacific coast to discuss implementation of the Observer Program--a
program designed to collect information about discarded fish for the
non-hake west coast groundfish fleets. [Footnote 17]
Finally, the Northwest Center now participates in NMFS' National Sea
Grant program to augment a Northwest Center-supported graduate study
program at the University of Washington that trains stock assessment
scientists. The Sea Grant program provides fellowships for students
interested in marine research, such as stock assessment methodology and
marine resource economics. The Northwest Center plans to employ two Sea
Grant students during the summer of 2004.
Further Actions Are Needed to Improve Stock Assessments:
While the Northwest Center has implemented some recommendations aimed
at improving stock assessments, it has not yet fully implemented many
others. These recommendations include collecting additional types of
data, such as ecosystem and recruitment information; improving data
quality, such as calibrating survey vessel equipment; and increasing
training opportunities for Northwest Center staff. Also, NMFS has not
acted on the task force recommendation to combine the improvement plan
and its complimentary plans into a comprehensive plan that provides
integrated program initiatives.
According to the improvement plan, additional types of data will allow
NMFS to further test and validate model assumptions, thereby increasing
the reliability of the stock assessments. The improvement plan further
states that information derived from ecosystem research and recruitment
surveys is essential if assessments are to meet the national standards
of "next generation" assessments or Tier 3 status. According to
Northwest Center officials, ecosystem information and coastwide
recruitment surveys are two of the most critical data sets needed to
ensure continuous improvement of groundfish stock assessments. The
Northwest Center conducts ecosystem research as part of its Science for
Ecosystem-based Management Initiative. Understanding the complex
ecological relationships between fish and the environment within which
they exist provides a better understanding of the effects of the
ecosystem on the groundfish fisheries and the scientific knowledge
needed to make informed ecosystem-based management decisions. Although
research is ongoing to develop ecosystem information, only a limited
amount of the data is collected and used in stock assessments. For
example, ecosystem data are collected during shelf and slope bottom
trawl surveys as time and resources allow. However, this information is
not widely incorporated into stock assessments. For the five species we
reviewed, only the boccacio assessment used ecosystem data--information
on the temperature of the ocean's surface.
According to Northwest Center officials, the collection of ecosystem
data is limited because the relatively small size of the commercial
vessels used in the shelf and slope bottom trawl surveys cannot support
the number of researchers needed to effectively conduct comprehensive
ecosystem research and collection activities. Furthermore, the
implementation of comprehensive ecosystem research and data collection
programs is contingent upon the funding of a dedicated research vessel
for west coast surveys. Northwest Center officials said they are to
receive a dedicated research vessel sometime during calendar years 2008
or 2009, at the earliest.
Better recruitment information for Pacific groundfish is also needed
because such information provides an early predictor of fish abundance,
especially for species such as hake, where there is a great variation
in recruitment. Northwest Center officials said that current
recruitment surveys are limited because existing funds support only
yearly surveys in selected areas. To achieve the best early predictions
of stock status, these officials said, recruitment surveys should be
coastwide and conducted twice a year. According to Northwest Center
officials, 13 full-time staff are needed to expand these and other
high-priority data collection efforts, such as surveys in untrawlable
habitat and expanded acoustic surveys.
The lack of quality data was identified in the improvement plan as an
impediment to producing reliable stock assessments. For example, when
equipment on different survey vessels are not calibrated, the data are
not comparable, and trends may not be accurately determined. The
Northwest Center is continuing its efforts to calibrate survey vessel
equipment.
The improvement plan also recommended that NMFS provide additional
training to ensure that qualified NMFS staff are available now and in
the future to conduct stock assessments and related activities. For
example, the plan recommended the development of a comprehensive
training program and more professional developmental opportunities for
NMFS' scientific staff. Northwest Center officials said they try to
meet the training and professional development needs of their
scientific staff. However, to date they have focused on developing
external training programs, such as the University of Washington
graduate program, to develop stock assessment scientists for the future
and have yet to develop a comprehensive training program for in-house
stock assessment scientists.
Finally, the improvement plan recommended that NMFS prepare a
comprehensive plan that combines the improvement plan with other
complementary plans, such as the NOAA Fisheries Data Acquisition Plan
and the NMFS Social Sciences Plan. A comprehensive plan would allow
NMFS to better integrate and coordinate program initiatives for
improving stock assessments. For example, the acquisition plan--the key
complementary plan to the improvement plan--identifies the need for
fishery research vessels to satisfy NMFS' data collection needs.
Although the improvement plan includes the number of staff that would
participate in data collection surveys, it does not contain the capital
and operating costs of the research vessels. Similarly, the staffing
requirements for augmenting the social sciences capabilities of NMFS to
conduct economic analyses is represented in the sciences plan and not
in the stock improvement plan. A NMFS official said that other program
priorities, such as conducting more stock assessments and improving
data collection activities, have precluded them from developing a
comprehensive plan.
Remaining Improvements Estimated to Cost at Least $8.9 Million:
According to NMFS funding and budget requests, the Northwest Center
needs at least $8.9 million to complete ongoing and planned
improvements to the stock assessments for Pacific groundfish. However,
the actual cost of implementing remaining improvements to Pacific
groundfish stock assessments may be even higher because the Northwest
Center's budget requests primarily reflect the amount of money the
Center believed it could realistically obtain, rather than the actual
cost of the improvements.
Remaining Improvements Are Estimated to Cost at Least $8.9 Million, but
Estimate Is Likely Understated:
According to NMFS, the Northwest Center needs at least $8.9 million to
complete ongoing and planned improvements for Pacific groundfish stock
assessments: $2.6 million that NMFS' Northwest Center requested but did
not receive between fiscal years 2001 to 2003 and $6. 3 million the
Center requested for fiscal years 2004 and 2005. Specifically, as shown
in table 2, the Northwest Center records have identified the following
funding needs:
* $7.7 million to improve the types of data used, including $2.4
million for surveys of untrawlable waters, $2.1 million to expand
acoustic and recruitment surveys, and $3.2 million to collect ecosystem
data; and:
* $1.2 million to improve the quality of data used in stock
assessments, including $600,000 to enhance the calibration of vessel
equipment; $525,000 to develop and implement methods to collect
information on stock identification, structure, and movement; and
$75,000 to standardize trawl survey procedures.
Table 2: Pacific Groundfish Stock Assessment Improvement Funds
Requested and Received by Data Type and Quality, Fiscal Years 2001-
2005:
Dollars in millions.
Types of data used: Surveys for untrawlable waters: Nearshore surveys
(2003);
Requested: $1.10;
Funded: $0.50;
Unfunded requests: $0.60.
Types of data used: Surveys for untrawlable waters: Untrawlable habitat
surveys[A] (2003);
Requested: $0.30;
Funded: $0.20;
Unfunded requests: $0.10.
Types of data used: Surveys for untrawlable waters: Longline groundfish
fishing tag surveys[A] (2003);
Requested: $0.40;
Funded: $0;
Unfunded requests: $0.40.
Types of data used: Surveys for untrawlable waters: Augment trawl
survey in nearshore (2004);
Requested: $0.51;
Funded: N/A;
Unfunded requests: $0.51.
Types of data used: Surveys for untrawlable waters: Fixed-gear survey
(2004);
Requested: $0.78;
Funded: N/A;
Unfunded $0.78.
Subtotal;
Unfunded requests: $2.4.
Types of data used: Expand acoustic and recruitment surveys:
Recruitment surveys (2003);
Requested: $0.40;
Funded: $0.20;
Unfunded requests: $0.20.
Types of data used: Expand acoustic and recruitment surveys: Acoustic
studies of Southern California bight rockfish[A] (2003);
Requested: $0.20;
Funded: $0;
Unfunded requests: $0.20.
Types of data used: Expand acoustic and recruitment surveys: Coastwide
augmentation of recruitment survey by Southwest Fisheries;
Science Center (2004);
Requested: $0.45;
Funded: N/A;
Unfunded requests: $0.45.
Types of data used: Expand acoustic and recruitment surveys: Acoustic
surveys in inaccessible habitat (2005);
Requested: $1.25;
Funded: N/A;
Unfunded requests: $1.25.
Subtotal;
Unfunded requests: $2.1.
Types of data used: Ecosystem data: Coastwide observing system (2003);
Requested: $0.80;
Funded: $0.60;
Unfunded requests: $0.20.
Types of data used: Ecosystem data: West coast observing system[A,B]
(2003);
Requested: $0.46;
Funded: $0.20;
Unfunded requests: $0.26.
Types of data used: Ecosystem data: Habitat-specific resource surveys
and fishing gear impacts (2004);
Requested: $1.45;
Funded: N/A;
Unfunded requests: $1.45.
Types of data used: Ecosystem data: Ecosystem studies of ocean
productivity and climate impacts (2004);
Requested: $1.28;
Funded: N/A;
Unfunded requests: $1.28.
Subtotal;
Unfunded requests: $3.2.
Total;
Unfunded requests: $7.7.
Quality of data used: Fishing survey vessel calibration (2003);
Requested: $0.60;
Funded: $0;
Unfunded requests: $0.60.
Quality of data used: Trawl standardization (2003);
Requested: $0.45;
Funded: $0.38;
Unfunded requests: $0.08.
Quality of data used: Stock identification, structure, and movement
(2004);
Requested: $0.53;
Funded: N/A;
Unfunded requests: $0.53.
Total;
Unfunded requests: $1.2.
Other: Stock assessment training and coordination (2003);
Requested: $0.35;
Funded: $0.15;
Unfunded requests: $0.20.
Other: Stock assessment training[A,B] (2003);
Requested: $0.30;
Funded: $0.50;
Unfunded requests: $-.20.
Other: Stock assessment training (2003);
Requested: $0.29;
Funded: $0.29;
Unfunded requests: $0.
Other: Stock assessment training (2002);
Requested: $0.29;
Funded: $0.29;
Unfunded requests: $0.
Other: Stock assessment training (2001);
Requested: $0.29;
Funded: $0.29;
Unfunded requests: $0.
Total;
Unfunded requests: $0.
Total unfunded requests;
Unfunded requests: $8.9.
Source: GAO analysis of NMFS data.
Notes: These figures exclude the cost of constructing a new fishing
research vessel, which is currently designated to be home ported at and
used primarily by the Northwest Center for surveys of fish species,
such as Pacific groundfish. This vessel has not yet been funded and
construction has not yet begun.
The amounts requested by the Northwest Center in fiscal years 2004 and
2005 are preliminary budget requests.
[A] Includes funding for the Southwest Fisheries Science Center
assistance to the Northwest Center's Pacific groundfish stock
assessment improvement program.
[B] Represents requests and funding for all marine species--not Pacific
groundfish alone. NMFS was unable to identify what portion of these
funds went solely to groundfish.
[End of table]
The Northwest Center did not receive its full funding request, in part,
because NMFS did not receive all the funding it had requested. Between
fiscal years 2001 and 2003, NMFS received $20.6 million (80 percent of
its request) in additional funding to implement improvements for all
marine stock assessments. NMFS allocated $3.6 million (58 percent of
funds the Northwest Center requested) to the Northwest Center for
improving Pacific groundfish stock assessments, resulting in a $2.6
million shortfall in the Center's request. This shortfall occurred in
part because of NMFS' need to balance the requests of its six science
centers against its program priorities and the available
funds. [Footnote 18] According to NMFS officials, their goal is to
achieve parity among the science centers in terms of their capability
to conduct scientific work, such as stock assessments.
Estimated Costs May Understate Actual Cost of Remaining Improvements:
The $8.9 million needed to implement remaining recommended improvements
is probably understated because the Northwest Center's budget requests
primarily reflect the amount of money the Center officials believed
they could realistically obtain, rather than the amount the
improvements would actually cost, according to NMFS officials. The
Northwest Center's budget requests for fiscal years 2004 and 2005 are
preliminary requests submitted before the Northwest Center received
its fiscal year 2003 funding. [Footnote 19] Consequently, the Northwest
Center will likely submit revised budget requests for fiscal years 2004
and 2005 that account for both its unfunded needs from fiscal years
2001 through 2003 and items that were unexpectedly funded in fiscal
year 2003. Moreover, the fiscal year 2004 and 2005 preliminary budget
requests do not incorporate any unanticipated problems or data gaps
that have developed since the Northwest Center submitted its
preliminary requests.
According to NMFS officials, NMFS' science centers, including the
Northwest Center, primarily make and justify their funding requests in
response to how much money Congress appropriates. After Congress passes
NMFS' budget, NMFS asks its science centers to reassess and detail how
much new money each needs to implement science center programs, such as
marine stock assessment improvements. According to NMFS officials, it
is unrealistic for a science center to request more funds than are
available in its appropriation, even if it needs more. While NMFS'
Northwest Center requested $6.2 million to implement improvements to
Pacific groundfish stock assessments between fiscal years 2001 and
2003, NMFS' 2001 West Coast Groundfish Research Plan estimated that
almost twice as much money would be needed--approximately $11.7 million
in new funding--to implement top-priority improvements to Pacific
groundfish stock assessments.
NMFS is now updating its plan and cost estimates for improving Pacific
groundfish stock assessments. Using key findings from its December 2003
review of the groundfish program, the Northwest Center plans to update
its groundfish research plan, last published in 2001. According to
NMFS, the updated groundfish research plan should be completed in late
2004 and is designed to (1) provide a comprehensive framework for
Pacific groundfish, (2) identify some of the greatest information gaps,
and (3) provide guidance for setting priorities on work to fill these
gaps. In addition, the updated plan will estimate how much such
improvements will cost.
Conclusions:
Stock assessments are the key to effectively managing fisheries. They
provide estimates of the species population, which NMFS uses to set
harvest limits that allow for sustainability and/or recovery of the
species. While stock assessment results often change from assessment
report to assessment report, the more types of information used in the
assessments, such as recruitment surveys and ecosystem studies, and the
greater the accuracy and quality of the data, such as scientifically
designed and collected data, the more reliable the assessment results.
However, the Pacific groundfish assessments we reviewed did not (1) use
scientifically designed and collected NMFS data of sufficient scope and
accuracy, such as survey data on the abundance of groundfish residing
in rocky, untrawlable habitats; (2) subject the non-NMFS data used to a
standard process for assessing its reliability; and/or (3) identify the
uncertainties of the assessments total biomass estimates. As a result,
the reliability of the five assessments is questionable. Without
reliable assessments, fishery managers may reach erroneous conclusions
and take actions that could adversely affect the fishing industry
economically or adversely affect the recovery and sustainability of the
fishery resources. Moreover, without a comprehensive, integrated
improvement plan, funding requests and planned actions to improve the
stock assessments may not be coordinated, jeopardizing successful and
timely implementation of assessment improvements.
Recommendations for Executive Action:
To improve the reliability of Pacific groundfish stock assessments, we
recommend that the Secretary of Commerce require the Director of
National Marine Fisheries Service to take the following four actions:
* Continue efforts to collect more types of data, such as data obtained
from surveys in rocky, untrawlable waters, recruitment surveys, and
ecosystem studies, for groundfish assessments where reliable data are
now lacking.
* Establish a standard approach that requires that non-NMFS data used
in stock assessments be evaluated for its reliability, and continue
efforts to implement the task force's recommendations to improve data
quality.
* Require that stock assessment reports clearly present the
uncertainties in the assessments, such as the margin of error
associated with species biomass estimates.
* Develop a comprehensive plan that integrates the NMFS stock
assessment improvement plan with other NMFS plans to ensure that stock
assessment improvement actions and budget requests are coordinated.
Agency Comments and Our Evaluation:
We provided the Department of Commerce with a draft of this report for
review and comment. We received a written response from the Under
Secretary of Commerce for Oceans and Atmosphere that included comments
from the National Oceanic and Atmospheric Administration (NOAA). NOAA
generally agreed with the report's accuracy and concurred with the
reports recommendations. However, NOAA said it was concerned about the
report's conclusion--that the reliability of stock assessments is
questionable for the five species reviewed--because it could be
misconstrued to infer that the assessments are unreliable for use in
managing the west coast groundfish fishery. In this regard, NOAA
provided additional comments to show the usefulness of the assessments,
even if some of the input data used in the assessments contained
errors. We stand by our conclusions that the five stock assessments we
reviewed were questionable because the input data were insufficient
and/or potentially inaccurate and that four of the assessment reports
did not present the uncertainties associated with the biomass
estimates. Nonetheless, we added language to the report to address
NOAA's concern. Specifically, we expanded upon the fact that NMFS used
the best information available at the time the stock assessments were
conducted by adding information on the importance of the assessments to
effectively manage the fisheries. Without these stock assessments, NMFS
and fishery managers would have very limited information on which to
base fishery management decisions.
NOAA agreed with the report recommendation to continue collecting more
types of data for groundfish assessments where reliable data are now
lacking. NOAA said that the reliability of stock assessments will be
improved if NMFS survey efforts are expanded and additional NMFS
fishery data are collected. NOAA said NMFS places a priority on these
improvements and will continue efforts to address this and other
recommendations to improve the collection of fishery data as funding
becomes available.
NOAA also agreed with the report recommendation to establish a standard
approach to evaluate the reliability of non-NMFS data used in stock
assessments and continue efforts to improve data quality. NMFS said,
through its west coast fishery science centers, it participates on
interagency data committees, to develop quality assurance protocols and
to assess the quality of non-NMFS data. NOAA agreed that it is
important to ensure that these interagency data committees continue to
highlight the need for standardized quality control procedures for the
collection of data.
NOAA agreed with the report recommendation to clearly present the
uncertainties in the stock assessments. NOAA said that quantifying
uncertainty of stock assessments is important to sound decision-making
by providing more information about the assessment, although this
quantification does not reduce the uncertainty in the assessment
itself. While the methods used and the completeness of the uncertainty
characterization varied from assessment to assessment, NOAA said it is
desirable to have both a quantitative analysis of model uncertainty and
an evaluation of the consequences of alternative model scenario.
Finally, NOAA agreed with the report recommendation to develop a
comprehensive plan that integrates the stock assessment plan with other
NMFS plans to ensure that improvement actions and budget initiatives
are coordinated. NOAA said that while much remains to be done, long-
term planning efforts and coordination among field and headquarters are
ongoing, and NOAA is committed to these actions.
NOAA's comments and our detailed responses are presented in appendix II
of this report. NOAA also provided technical comments that we
incorporated in this report as appropriate.
As agreed with your offices, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 7 days
from the report date. At that time, we will send copies of this report
to the Secretary of Commerce and the Director of the National Marine
Fisheries Service. We will also provide copies to others upon request.
In addition, the report will be available at no charge on the GAO Web
site at [Hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please call
me at (202) 512-3841 or Keith Oleson at (415) 904-2218. Key
contributions to this report are listed in appendix III.
Signed by:
Anu K. Mittal,
Director, Natural Resources and Environment:
[End of section]
Appendixes:
Appendix I: Objectives, Scope, and Methodology:
We reviewed National Marine Fisheries Service (NMFS) stock assessments
for five species of Pacific groundfish: Pacific hake (Pacific whiting)
as well as four types of rockfish--bocaccio, canary, darkblotched, and
yelloweye. Specifically, for these five species you asked us to (1)
assess the reliability of NMFS' stock assessments, (2) identify which
relevant recommendations from the stock assessment improvement plan
have been implemented and which have not, and (3) identify the
estimated costs associated with planned and ongoing improvements to
groundfish stock assessments. We did not review the stock assessments
of any of the other west coast Pacific groundfish species, thus the
information contained in this report pertains to the five species we
reviewed unless stated otherwise.
For all three objectives, we reviewed key laws and agency reports and
interviewed officials from NMFS, including officials from the Northwest
Fisheries Science Center, which has lead responsibility for conducting
Pacific groundfish stock assessments with assistance from other west
coast science centers.
To assess the reliability of the Pacific hake, bocaccio, canary,
darkblotched, and yelloweye stock assessments, we examined
methodological and administrative documents developed by NMFS and
others to support the groundfish data collection, maintenance, and
assessment process. We reviewed the controls over stock assessment
data, the types of fish population surveys used, and recent Pacific
groundfish stock assessment studies (2002 and 2004 studies for Pacific
hake, 2002 and 2003 studies for bocaccio, 2002 study for canary, 2000
and 2003 studies for darkblotched, and 2001 and 2002 studies for
yelloweye). We examined whether and to what extent NMFS has processes
and procedures in place to ensure the reliability of data used in the
Pacific groundfish stock assessments. We reviewed the stock assessment
reports and determined whether they articulated the level of
uncertainty in the assessment model estimates. We interviewed an array
of government officials and fisheries experts, including the Oregon
Department of Fish and Wildlife, the Washington Department of Fish and
Wildlife, the California Department of Fish and Game, the Pacific
Fishery Management Council, the Pacific States Marine Fisheries
Commission, environmental groups, and industry associations, as well as
fishermen and academics. We did not simulate NMFS' stock assessment
models nor evaluate the mathematical and statistical methodologies used
in the models for Pacific hake, bocaccio, canary, darkblotched, and
yelloweye.
To identify the relevant recommendations to improve stock assessments
that NMFS has implemented and has not implemented, we reviewed agency
reports on marine fisheries stock assessments, strategic planning, and
data collection. We also interviewed officials from the Oregon
Department of Fish and Wildlife, the Washington Department of Fish and
Wildlife, the California Department of Fish and Game, the Pacific
Fishery Management Council, and the Pacific States Marine Fisheries
Commission as well as environmental groups, industry associations,
fishermen, and academics.
To determine the estimated costs associated with NMFS' planned and
ongoing improvements to Pacific groundfish stock assessments, we
reviewed relevant budget requests and funding documents for fiscal
years 2001 through 2005 and interviewed National Oceanic and
Atmospheric Administration officials. We did not evaluate the accuracy
of NMFS' budget requests for specific project items but rather used the
amounts NMFS requested for these project items to estimate the total
additional costs of implementing the planned and ongoing improvements
to Pacific groundfish stock assessments.
We conducted our review from May 2003 through April 2004 in accordance
with generally accepted government auditing standards.
[End of section]
Appendix II: Comments From the Department of Commerce:
UNITED STATES DEPARTMENT OF COMMERCE
The Under Secretary of Commerce for Oceans and Atmosphere
Washington, D.C. 20230:
MAR 13 2004:
Ms. Anu K. Mittal:
Director, Natural Resources and Environment:
United States General Accounting Office:
441 G Street, NW:
Washington, D.C. 20548:
Dear Ms. Mittal:
Thank you for the opportunity to review and comment on the General
Accounting Office's draft report entitled "Pacific Groundfish:
Continued Efforts Needed to Improve Reliability of Stock Assessments,"
GAO-04-606. Enclosed is the National Oceanic and Atmospheric
Administration's comments on the draft report.
These comments were prepared in accordance with the Office of
Management and Budget Circular A-50.
Sincerely,
Signed by:
Conrad C. Lautenbacher, Jr. Vice Admiral, U.S. Navy (Ret. )
Under Secretary of Commerce for Oceans and Atmosphere:
Enclosure:
NOAA Comments on the Draft GAO Report Entitled "Pacific Groundfish:
Continued Efforts Needed to Improve Reliability of Stock Assessments"
(GAO-04-606/June 2004):
Comment on the Scope of the Report:
The report title implies the scope of this report is Pacific Coast
wide, but the focus is heavily weighted to survey and assess activities
of the Northwest Fisheries Science Center (NWFSC). The report uses the
acronyms NMFS and NWFSC interchangeably. Inclusion of information from
the Southwest and the Alaska Fisheries Science Centers is highly
inconsistent. Insufficient information was included to fully portray
the scope, strengths and weaknesses of their past and present
contributions to Pacific Coast groundfish work. However, inclusion of
bocaccio in the set of GAO-reviewed stocks and references to NWFSC and
AFSC work were frequent enough to preclude casting the report as
exclusively pertaining to the NWFSC. We recommend the report be
characterized as "A review of the NWFSC's groundfish program including
aspects of West Coast groundfish research conducted by other West Coast
Science Centers." This is more accurate than referring to it either as
a review of the NWFSC or NOAA Fisheries' West Coast Groundfish program.
It is more than the former, but less than the latter. This scope should
be made clear in the introduction of the report.
Recommended Changes for Factual Information:
GAO Highlights, par. 1, line 9: Replace 2002 with 2003.
GAO Highlights, par. 1, lines 14-16: Change to read "The assessors who
reviewed the quality of the non-NMFS data found errors that made "some
of the data unusable or impaired the reliability of "certain" stock
assessments." See our response to GAO recommendation 2. Also, this
statement does not pertain to the hake assessment.
GAO Highlights, par. 2, line 10: This statement is not correct. NMFS
has increased a limited number of recruitment surveys and work on
untrawlable habitat.
Page 1, par. 2: NMFS also uses assessments conducted by independent
scientists in state resource agencies.
Page 1, par. 2, line 4: Should read, "Northwest Fisheries Science
Center," as statement is not correct for NMFS generally.
Page 1, par. 2, line 5: ...(1) the shelf and slope [insert "bottom"]
trawl survey...
Page 2, par. 1, line 1: Delete "In response," replace with "In 1999,"
...review the stock assessment process "across the agency."
Page 2, par. 1, line 4: ...for conducting "some of the" Pacific ...
Page 2, par. 1, line S: Add citation for October 2001 Marine Fisheries
Stock Assessment Improvement Plan NMFS. 2001. Marine Fisheries Stock
Assessment Improvement Plan. Report of the National Marine Fisheries
Service National Task Force for Improving Fish Stock Assessments,
Second Ed. (Rev. ). U.S. Dep. Commerce, NOAA Tech. Memo. NMFS-F/SPO-56,
69 p. , 25 appendices.
Page 2, par. 3, line 2: The publication date for the canary assessment
was 2002.
Page 3, par. 1, line 4: Add "1998" National Research Report*, and
footnote citation: "*NRC 1998. Improving Fish Stock Assessments.
National Research Council. National Academy Press. Washington, D. C. 177
p."
Page 3, par. 1, lines 7-11: These statements are incorrect because NMFS
had and used fishery-independent data for all but the yelloweye
rockfish assessment. Revision to this page should be made in line with
the general comments we have made regarding use of surveys in
assessments.
Page 3, par. 1, line 13: Replace 2002 with 2003.
Page 4, par. 2: There is no clarification of budget needs as NMFS-wide
versus NWFSC-specific.
Page 6, par. 2, bullets 3 and S, and footnote 2: The NWFSC and SWFSC
have joint responsibility for salmon fishery and ESA responsibility in
freshwater rivers and streams in Washington ...etc. and California.
Page 8, bullet 1: Stock abundance ...stock's total "size or weight."
Delete "population."
Page 8, bullet S: Recruitment research ...abundance of juvenile "and
larval" fish...
Page 8, paragraph after bullets: The last phrase in the sentence is not
correct. It should read, "For each stock assessment, a review panel,
consisting of NMFS scientists and outside experts, independently
reviews the methodology of the assessment and works with the assessor
to ensure their comments are adequately addressed."
Page 8, last par. , line 3: Seven species of rockfish have been declared
overfished. The other three are cowcod, widow rockfish, and Pacific
Ocean perch.
Page 9, bullet 1, line 2: Replace "11 other species of hake" with "15
species of the genus Merluccius (hakes)*" [could include a new
reference footnote - *Source: www. fishbase. org"]
Page 12, line 14; also page 14, full par. 1: Our general comments
describe how the data situation for these species is not nearly as
limited as described here. Certainly bocaccio has more fishery-
independent data than acknowledged here. Obtaining data from
untrawlable habitat will improve the precision of the canary, bocaccio
and darkblotched assessments and, perhaps more importantly, it will
enable assessments for other rockfish that occur primarily on these
habitats.
Page 13, Table 1 and par. 1: Bocaccio clearly rates a "yes" for NMFS-
collected data. The Ca1COFI larval abundance data has been collected on
a rigorous scientific basis by NMFS (SWFSC) for over 50 years. In
addition, our general comments clarify the degree to which the survey
data are not critically limited for darkblotched and canary rockfish.
Page 13, Table 1: Highly standardized protocols are used for collecting
the fishery-dependent data, including the non-NMFS data for rockfish.
Our general comments show at least hake should be rated as a "yes" in
this category.
Page 13, par. 1, line 3: ...slope [insert "bottom trawl"] survey...
Page 13, par. 1, lines 4-S: The comma in the phrase "include, larval"
should probably be deleted. The phrase "The non-NMFS data assessors use
include...." is slightly awkward. Could it be replaced with "Assessors
use non-NMFS data such as..."?
Page 14, full par. 1, line 10: Replace 2002 with 2003.
Page 15, par. 1, line 1: Replace "lives in a different habitat than the
other four species," with "live primarily in mid-water, above the sea
floor, and so..."
Page 15, footnote 9: Reposition on page 14; replace citation as
follows: Zimmerman, M. 2003. Calculation of untrawlable areas within
the boundaries of a bottom trawl survey. Can. J. Fish. Aquat. Sci.
60:657-669.
Page 16, full par. 1, line 2: Introduces the NMFS 2001 Stock Assessment
Improvement Plan in the document text. It would be helpful to also
introduce the acronym "Marine Fisheries SAIP" and then use in the
following pages. There are many references after this point using the
phrase "improvement plan" and it seems likely all of these refer to the
SAIP, but there also is the West Coast Groundfish Research Plan.
Page 16, full par. 2, line 2: Replace "health" with "stock abundance."
Page 16, footnote 10: Reposition on page 15.
Page 17, last par. , line 3: ...tiers of assessment [insert
"improvements"] are as...
Page 19, par. 1, line 2: ...frequent acoustic [insert ","] and shelf
and slope [insert "bottom trawl"] surveys.
Page 19, par. 1: While this list of actions may accurately reflect what
the NWFSC did to improve data quantity, it is not a comprehensive list
of all that has been done to improve data quantity coast wide, and
especially omits projects initiated by the SWFSC, some joint with the
NWFSC.
Page 21, par. 1: While this may represent the NWFSC's ecosystem science
programs, it is not an accurate portrayal of the coast wide program.
For example, in the SWFSC, both the Ca1COFI survey and the pelagic
juvenile rockfish survey collect comprehensive environmental/ecosystem
data, and the SWFSC's Pacific Fisheries Environmental Laboratory is
wholly devoted to the conduct of environmental research in a fisheries
context.
Page 21, par. 2, line S: Replace "is" with "are."
Page 21, par. 2, line 6: Substitute ...dedicated research vessel for
West Coast surveys.
Page 24, Table 2: Footnote b only refers to requests and funding for
the SWFSC. The NWFSC separated groundfish funding from non-groundfish
funding.
Page 26, last sentence of first full paragraph: This is somewhat
misleading. The groundfish research plan was developed with all the
NMFS Centers as a coastwide plan. So the term "its" in this sentence is
misleading. In the next paragraph, the coast wide research plan will be
updated (not just the NWFSC plan).
Page 26, par. 4, line 3: Suggest replacing "survivability" with
"sustainability."
Page 27, par. 1, line 4: (1)...etc. See comment for page 12 above.
Page 27, par. 1, line I1: "...survivability of the fishery
["resources"? or " industry" ?]
General Comments:
Reliability of Assessments:
The GAO report states, "The reliability of NMFS stock assessments is
questionable for the Pacific hake and four rockfish species we
reviewed, although the assessments were based on the best information
available at the time they were conducted." This statement does not
adequately convey the different degrees of precision associated with
each of these assessments and could easily be misconstrued to mean all
these assessments are an unreliable basis for management of the West
Coast groundfish fishery. NOAA doubts GAO intended such an extreme
conclusion from their findings, and NOAA would like to take this
opportunity to provide additional information regarding these
assessments: namely, that all have passed a scientific review and all
are serving as the basis for formal status determination and management
of these stocks. Our response to GAO recommendation 2 demonstrates
quality assurance for non-NMFS data is not absent; thus, the
assessments are not unreliable due to use of these data. Our response
to GAO recommendation 3 indicates precision is quantified to the extent
feasible today, is on par with state-of-the-science assessments
elsewhere, and is rapidly improving. NOAA acknowledges quantifying
precision of these assessment results would provide more information
about the assessment, and expanded and improved data would improve the
certainty of the assessment results and forecasts. Nevertheless, there
is no reason to believe these assessments are inadequate for guiding
this fishery. In addition to NOAA's responses to GAO recommendations 2
and 3, NOAA provides the following information with regard to the
reliability of these stock assessments:
Surveys as Stock Trend Indicators:
Stock. assessments require data on the relative trend in stock
abundance and on the removals (catch). Both are needed in order to
produce the target assessment results. Although catch data alone are
not a substitute for survey trend data, in some circumstances with
fishery effort information, the catch per unit of fishing effort can be
standardized to provide an index of the trend in stock abundance. The
National Research Council's report on "Improving Fish Stock
Assessments" and NOAA's own studies indicate stock assessments should
include a standardized, scientifically-designed survey of fish
abundance so that changes observed in the survey confidently reflect
true changes in the stock's abundance and not some confounding factor.
These conditions are better met with the controlled conditions of a
fishery-independent survey than with fishery catch per effort.
NMFS and Non-NMFS Data:
The GAO report categorized the five reviewed assessments according to
the availability of NMFS-collected survey data, often abbreviated to
just "NMFS data." While NMFS collects nearly all the fishery-
independent survey data for West Coast groundfish and the states
collect much of the fishery-dependent data on catch and fishing effort,
it is more pertinent to focus on the degree of standardization of the
survey data than on the source. NMFS is firmly committed to providing
such surveys on a long-term basis, but useful surveys can be conducted
by non-NMFS organizations. The desire for standardized, fishery-
independent surveys should not be equated with a concern about basing
an assessment on non-NMFS data. Also, less precise assessments can be
conducted with only non-NMFS fishery-dependent data as long as there is
adequate standardization and quality assurance.
Assessments:
The Pacific hake assessment is one of the most data-rich and
sophisticated assessments in the nation. It is based upon a highly
calibrated acoustic survey of hake abundance and a very complete system
for monitoring the hake catch. Even if the portion of the hake catch
data provided by the states had some major shortcomings (which is
unlikely), this would have little impact on the overall assessment's
precision and certainly would not make it unreliable. More frequent
Pacific hake acoustic surveys and a more geographically extensive
recruitment survey will improve the timeliness and precision of short-
term forecasts, but the overall assessment of the status of the stock
certainly is highly reliable according to all scientific reviews.
The bocaccio, canary, and darkblotched rockfish assessments all obtain
adequate abundance trend information from the NMFS bottom trawl surveys
because these species commonly occur in these surveys. The bocaccio
assessment includes several other fishery-independent sources of
abundance trends, particularly the NMFS-SWFSC juvenile rockfish survey
providing an index of recruitment, and the SWFSC led CaICOFI survey
provides a larval index of spawning biomass. The fact that the bottom
trawl survey cannot access the roughest habitat does not invalidate its
usefulness as an index of relative changes in the overall abundance of
these three stocks, even if the density of fish in untrawlable habitat
is typically different than the density on trawlable habitat assuming a
random or consistent bias. New surveys, some based on new technology,
providing information from untrawlable habitat will allow more direct
evaluation of the total
stock and calibration of the bias assumptions, but the current fishery-
independent survey data are likely to be sufficiently complete to
determine these stocks have declined substantially into an overfished
state. With improved and more frequent surveys and more timely
assessments using these surveys, recovery of these stocks and possible
declines in other stocks will be more accurate and precise and detected
sooner.
The yelloweye rockfish assessment cannot obtain relevant information
from the NMFS bottom trawl survey because this species primarily occurs
on rough habitat; instead, the yelloweye assessment relies upon the
trend observed in recreational fishery catch per unit effort. Certainly
fishery catch per effort is less standardized a data source than the
NMFS bottom trawl survey so this is logically the least precise of the
assessments reviewed by GAO, but use of fishery catch per effort is a
standard practice for many important assessments where fishery-
independent surveys are impractical (such as highly migratory tunas).
The yelloweye assessment contains a caveat regarding uncertainty in the
calibration of the fishery catch per effort and it acknowledges the new
effort to collect new kinds of survey data from the untrawlable
habitat. The yelloweye assessment represents a rational use of
available fishery-dependent data until sufficient new fishery-
independent data become available.
On page 16, the GAO report references the doubling of estimated
bocaccio biomass when the assessment was redone in 2003 and reiterates
GAO's call for uncertainty ranges. Note, most of the doubling was due
to factors (a new abundance index and a change in the level of natural
mortality used in the assessment model) that would not be addressed in
a standard statistical analysis of uncertainty.
NOAA Response to GAO Recommendations:
Recommendation 1: "Continue efforts to collect more types of data, such
as data obtained from surveys in rocky, untrawlable waters, recruitment
surveys, and ecosystem studies, for groundfish assessments where
reliable data are now lacking."
NOAA Response: NOAA agrees with this recommendation. The reliability of
stock assessments will be improved if NMFS survey efforts are expanded
and additional fishery-independent data are collected. NMFS places a
priority on these improvements and through the Expand Stock Assessment
- Improve Data Collection budget line has already started new budget
initiatives and programmatic expansions in this area. The agency will
continue to address the recommendations of the Marine Fisheries Stock
Assessment Improvement Plan (and others) as funding becomes available.
Recommendation 2: "Establish a standard approach that requires that
non-NMFS data used in stock assessments be evaluated for its
reliability, and continue efforts to implement the task force's
recommendations to improve data quality."
NOAA Response: NOAA agrees with this recommendation. NMFS, through the
West Coast Fisheries Science Centers, participates on the interagency
data committees for the Pacific States Marine Fisheries Commission's
Pacific Fisheries Information Network (PacFIN) for commercial West
Coast fishery data, and the Recreational Fisheries Information Network
(RecFIN), to:
develop quality assurance protocols and to assess the quality of
fishery-dependent data. Quality assurance protocols were last outlined
and specified in a NOAA Technical Memorandum (Commercial Fisheries Data
Collection Procedures for the U.S. Pacific Coast Groundfish. 1997.
Sampson and Crone (eds)).
Each assessment relies upon these procedures and protocols to produce
sound fishery-dependent data with good quality assurance. In cases
where novel applications of these data are attempted, the assessment
scientists will be more involved in preparation of the raw data and may
find the data are inadequate for the' new application, such as catch
per unit effort for a particular fishery sector. Rejection of certain
data for such a new purpose does not invalidate the entire fishery
monitoring program. Therefore, NOAH agrees it is important to assure
the terms of reference of these interagency data committees continue to
highlight the need for standardized quality control procedures for the
collection of the data. NMFS membership on the PacFIN and RecFIN
committees assures such protocols are monitored and implemented.
Recommendation 3: "Require that stock assessment reports clearly
present the uncertainties in the assessments, such as the margin of
error associated with population estimates."
NOAA Response: NOAA agrees with this recommendation. Quantifying
uncertainty of assessments is important for sound decision-making by
providing more information about the assessment, although this
quantification does not reduce the uncertainty in the assessment
itself. Currently, the Scientific and Statistical Committee of the
Pacific Fishery Management Council has terms of reference for stock
assessments including requiring that the uncertainty be characterized.
However, the methods used and the completeness of the uncertainty
characterization has varied from assessment to assessment. It is
desirable to have both a quantitative analysis of model uncertainty due
to statistical goodness-of-fit within a selected model scenario, and an
evaluation of the consequences of alternative model scenarios to
investigate potential biases. Doing both for complex fishery stock
assessment models is technically challenging and the West Coast
Goundfish assessments are near the state-of-the-science. The GAO report
focused on the statistical goodness-of-fit, which was available for the
Pacific hake assessment, but this cannot be done routinely with the
older generation models necessary for the types of data available for
the four rockfish assessments reviewed by GAO. NMFS has only been able
to work slowly on new model development at its current level of
funding, but in 2004, it will roll out and review a next generation
model suitable for rockfish assessments that will be capable of
producing the desired uncertainty calculations. In addition, NMFS'
cooperative research with the University of Washington is exploring
methods to quantitatively combine uncertainty analyses across multiple
model scenarios, thus achieving a new level of overall uncertainty
characterization. NMFS will assist in developing new terms of reference
more clearly outlining the requirement for uncertainty characterization
and giving clear guidance on the methods to be used to calculate and
report this uncertainty.
Recommendation 4: "Develop a comprehensive plan that integrates the
NMFS stock assessment improvement plan with other NMFS plans to ensure
that stock assessment improvement actions and budget requests are
coordinated."
NOAA Response: NOAA agrees with this recommendation. The NMFS Office of
Science and Technology, together with all six regional Fisheries
Science Centers, is responsible for developing budget initiatives for
major program areas focusing on improving the scientific basis for
fishery management. Long-term program development and field-
headquarters' coordination activities are continuing for the following:
Expand Stock Assessments-Improve Data Collections, Fisheries
Statistics (National Fisheries Information System), Economics and
Social Sciences Research, Fisheries Observers, Fishery Oceanography,
National Cooperative Research, and new NOAA Fishery Survey Vessel fleet
construction. These multi-year program activities have been favorably
reviewed by our external partners, constituents, and the Congress.
Initial new funding has been received, but much remains to be done.
NOAA is committed to continuing on this course.
The following are GAO comments on NOAA's letter dated May 13, 2004.
GAO Comments:
1. We added clarifying language to the scope and methodology section of
the report to clearly identify the species and activities covered by
the review.
2. We revised the report to show the publication date.
3. We revised the report accordingly.
4. We revised the report to clearly show that NMFS has not collected
enough ecosystem data and that the frequency and range of recruitment
surveys are limited. The statement does not address untrawlable
habitat.
5. We revised the report to clarify that NMFS "generally uses" NMFS's
staff or contracts with outside experts.
6. We revised the report to more clearly differentiate between NMFS as
a whole and NMFS' Northwest Center in particular. We made similar
revisions, as appropriate, throughout the report.
7. We revised the report to specify "bottom" trawl survey. We made
similar changes, as appropriate, throughout the report.
8. We revised the report to include the year and scope of the task
force review.
9. We revised the report to indicate that the Northwest Center is
responsible for coordinating groundfish stock assessments.
10. We revised the report to include the citation.
11. We revised the report to include the date and citation of the
National Research Council report.
12. The NMFS data used in the bocaccio, canary, and darkblotched
assessments were limited because NMFS conducted its surveys in
trawlable waters only. NMFS data were not available for untrawlable
waters, which these species also inhabit. For this reason, we did not
revise the report.
13. We revised the report to clarify the shared responsibilities of the
Northwest and Southwest Centers.
14. We revised the report to include NOAA's recommended definition of
stock abundance.
15. We revised the report to include larval fish.
16. We revised the report to clarify the role of the review panel.
17. We revised the report to more explicitly distinguish the five
species related to our report from other overfished Pacific groundfish.
18. We revised the report to more clearly describe the distribution of
Pacific hake.
19. Bocaccio survey data for untrawlable habitats, as stated in comment
12, was unavailable. For this reason, we did not change the report.
20. NOAA commented that highly standardized protocols are used for
collecting non-NMFS data (fishery dependent data) for rockfish. We
found that although NMFS does have collection and quality assurance
procedures for state-collected non-NMFS data, NMFS does not check or
have a standard process to verify that these data have been reviewed
for reliability. As discussed in our report, some assessors chose to
review the raw data, while others did not. Assessors who voluntarily
reviewed raw non-NMFS data found mistakes that either made some of the
data unusable or could have impaired the accuracy of the stock
assessments. For these reasons, we did not change the report.
21. We revised the punctuation accordingly.
22. We revised the report to clarify that Pacific hake live in mid-
water habitat.
23. The footnote placement and citation are in accordance with GAO
guidelines. For this reason, we did not change the report.
24. We believe that our report has addressed this issue. By referring
to the West Coast Groundfish Research Plan by its complete title, we
adequately distinguish between the two reports. For this reason, we did
not change the report.
25. We changed "health" to "stock abundance.":
26. The footnote placement is repositioned in report.
27. We revised the report to include assessment "improvements.":
28. As indicated in our report, we illustrate some of the actions that
the Northwest Center took to improve data quantity and did not intend
to provide a comprehensive list of all actions conducted to improve
data quantity coast wide. However, we added footnote 16 to clarify the
actions taken by the Southwest Center.
29. The example we provided is not intended to be a comprehensive list
of all ecosystem research conducted on the west coast. Instead, it
illustrates the type of work the Northwest Center is conducting and the
opportunities for improving ecosystem research. For this reason, we did
not change the report.
30. After reviewing the report we believe no change is required because
of subject-verb agreement.
31. We added clarifying language.
32. We believe that table 2 notes "a" and "b" in our report already
adequately address this issue. Annotations for projects that do not
separate out groundfish funds occur only in items that are annotated as
Southwest Center projects. For this reason, we did not change the
report.
33. We changed "survivability" to "sustainability.":
34. We changed "survivability" to "sustainability" and added fishery
"resources" for clarification.
35. NOAA commented that GAO does not adequately convey the different
degrees of precision associated with the stock assessments and GAO's
conclusion that the reliability of the five assessments we reviewed is
questionable and could easily be misconstrued to mean all these
assessments are an unreliable basis for management of the west coast
groundfish fishery. NOAA also commented that the five assessments GAO
reviewed all passed scientific review and are serving as the basis for
formal status determination and fishery management. Our report
acknowledges that stock assessments are scientifically reviewed and are
a key tool for managing fisheries. However, we found the reliability of
the five assessments questionable for the three reasons we highlighted
in our report, and we recommended actions on how to improve the
reliability of the stock assessments. We added clarification to the
report to show that stock assessments are a key tool for managing
fisheries and are important in making decisions about setting harvest
levels and developing plans to rebuild overfished stocks.
NOAA also commented that quality assurance for non-NMFS data is not
absent. As stated in our response number 20, we found that although
NMFS does have collection and quality assurance procedures for state-
collected non-NMFS data, NMFS does not check or have a standard process
to verify that these data have been reviewed for reliability. As
discussed in our report, some assessors chose to review the raw data,
while others did not. Assessors who voluntarily reviewed raw non-NMFS
data found mistakes that either made some of the data unusable or could
have impaired the accuracy of the stock assessments. For these reason,
we did not change the report.
36. NOAA commented that it is more pertinent to focus on the degree of
standardization of the survey data than the source. By categorizing
data as NMFS data and non-NMFS data, we were not implying that non-NMFS
organizations could not conduct useful fishery-independent surveys. We
categorized the data in this manner because NMFS currently conducts
nearly all of the fishery-independent surveys and non-NMFS
organizations collect most of the fishery-dependent data. Footnote 7 in
the report states that NMFS generally refers to its data as fishery-
independent data and to non-NMFS data as fishery-dependent data. For
these reasons, we did not change the report.
37. We believe the Pacific hake biomass estimates are questionable
because the assessment used non-NMFS data that NMFS did not check or
subject to standard data reliability testing. Assessors who reviewed
raw non-NMFS data for other stock assessments found mistakes that
either made some of the data unusable or could have impaired the
accuracy of the stock assessments. For this reason, we did not change
the report.
38. NOAA commented that bocaccio, canary, and darkblotched assessments
all obtain adequate abundance trend information from the NMFS bottom
trawl surveys. NOAA also commented that although bottom trawl survey
cannot access the roughest habitat, it is useful as an index of
relative changes in the overall abundance. As stated in our report, we
found that the NMFS survey data used in these assessments were limited
in scope because the surveys were conducted only in trawlable areas.
Assessors estimated overall biomass using the NMFS data collected from
the trawlable area, which has a different abundance rate than the
untrawlable area. Stock assessors commented that relying on survey data
from trawlable waters only increases the uncertainty of stock
assessments. For these reasons, we did not change the report.
39. As noted in our report, the National Research Council found that
the inclusion of NMFS survey data was the best option for a reliable
estimate of abundance because such surveys use an unbiased statistical
design, control sampling locations, and provide for quality assurance.
Northwest Center officials said that to obtain reliable results, each
stock assessment should include at least one source of NMFS-collected
data of sufficient scope and accuracy because such surveys are unbiased
and scientifically designed. NMFS data were unavailable for the
yelloweye assessment. Northwest Center officials also raised concerns
about basing assessments solely on non-NMFS data such as commercial and
recreational catch data. Catch data do not provide the species'
relative or absolute biomass, according to NMFS officials. Catch data
alone are insufficient because fishermen are not randomly sampling the
ocean, but are fishing areas that they are allowed to fish and they
believe have the most fish; fishing restrictions, such as a total
allowable catch, can limit the amount of fish being caught; and catch
data have often been inaccurate for a variety of reasons, such as
imprecise accounting for dead fish tossed back into the ocean. For
these reasons, we did not change the report.
40. NOAA commented that the doubling of estimated bocaccio biomass in
2003 was due to factors that would not be addressed in a standard
statistical analysis. Although a standard statistical analysis may not
fully address the doubling of an estimate, an assessment without an
uncertainty range does not quantify and communicate any of the
uncertainty. For this reason, we did not change the report.
[End of section]
Appendix III GAO Contact and Staff Acknowledgments:
GAO Contact:
Keith W. Oleson, (415) 904-2218:
Staff Acknowledgments:
In addition to the person named above, Leo G. Acosta, Kristine N.
Braaten, Allen T. Chan, David Dornisch, Alan Kasdan, Robert Marek,
Cynthia C. Norris, Carol Herrnstadt Shulman, and Tama R. Weinberg made
key contributions to this report.
(360345):
FOOTNOTES
[1] Marine Fisheries Stock Assessment Improvement Plan, U.S. Department
of Commerce, NOAA Tech. Memo. NMFS-F/SPO-56, 69, 2001.
[2] Pub. L. No. 94-265, 90 Stat. 331 (codified as amended at 16 U.S. C.
§§ 1801-1883).
[3] In addition, the Northwest Center and the Southwest Center share
some responsibilities in freshwater rivers and streams in Idaho,
Oregon, Montana, and Washington.
[4] Ibid.
[5] Pub. L. No. 104-297, 110 Stat. 3559 (1996).
[6] Under the Magnuson-Stevens Act, "bycatch" means fish that are
harvested in a fishery, but which are not sold or kept for personal
use. Bycatch includes fish discarded for regulatory or economic
reasons.
[7] The 2003 darkblotched update was not a full stock assessment, but
rather updated the 2000 darkblotched stock assessment by adding some
additional data in order to estimate new biomass figures. The most
recent estimate from this update is for 2001.
[8] The most recent estimate of biomass for yelloweye from this stock
assessment is for 2001.
[9] NMFS generally refers to its data as fishery-independent data and
to non-NMFS data as fishery-dependent data.
[10] Science and Its Role in the National Marine Fisheries Service,
National Research Council, National Academy Press, July 2002.
[11] Zimmermann, Calculations of Untrawlable Areas within the
Boundaries of a Bottom Trawl Survey, NRC Research Press, July 2003.
[12] U.S. General Accounting Office, Standards for Internal Control in
the Federal Government, GAO/AIMD-00-21.3.1, (Washington, D.C.:
November 1999).
[13] Improving Fish Stock Assessments, National Research Council,
National Academy Press, 1998.
[14] Canary Rockfish STAR Panel Meeting Report, Northwest Fisheries
Science Center, May 7, 2002.
[15] U.S. General Accounting Office, Quantitative Data Analysis: An
Introduction, GAO/PEMD-10.1.11, (Washington, D.C.: May 1992).
[16] The Southwest Center also took some independent and joint actions
with the Northwest Center to improve data quantity.
[17] Pacific hake has a separate observer program. Discards are the
amount of fish unintentionally caught and not retained on the fishing
vessel. Discard information is collected for use in assessing the
mortality for a number of groundfish species.
[18] The sixth science center, the Pacific Islands Fisheries Science
Center, opened in 2003.
[19] In light of the delayed congressional appropriation of fiscal year
2004 funds, NMFS has not yet determined the precise amount of money
available to its science centers. As a result, as of April 2004, NMFS
had not yet asked the Northwest Center to update its preliminary
requests for fiscal year 2004 funds.
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www. gao. gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www. gao. gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D. C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www. gao. gov/fraudnet/fraudnet. htm E-mail: fraudnet@gao. gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao. gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D. C.
20548:
