Investigating the Accuracy and Robustness of

the Icelandic Cod Assessment and Catch

Control Rule

A. Rosenberg, G. Kirkwood, M. Mangel, S. Hill and G. Parkes

MRAG Americas, Inc.

Background

“Estimating the number of fish in the sea is just the same as counting the number of trees in a forest, except you can’t see the fish and they move.”

J. G. Shepherd

Total catch and national TACs 1977 to 2001. 1977 to 1990 calendar years; 1991 January to August; 1992 to date September to August.

Background to the Problem

0

100000

200000

300000

400000

500000

1975 1980 1985 1990 1995 2000 2005

ton

ne

s

Total Catch by Icelandfrom Iceland Grounds

National TAC

Background to the Problem• Up to 2000, stock assessments of Icelandic cod have been carried

out using the Extended Survivors Analysis (XSA) method. This method has been used extensively by many ICES working groups.

• The 2000 ICES assessment showed that the stock had been seriously overestimated in recent years. In May-June 2000 MRI asked a group of external experts led by Professor John Pope to review and re-analyse the assessment.

• The main conclusion of the review was that the MRI assessment (XSA using a number of CPUE tuning series from the survey and the commercial fleets) was overly optimistic with regard to estimating stock abundance in recent years, but that the methodology applied was essentially sound

• Several alternative methods gave results with lower abundance in recent years, but all of the methods show similar patterns in estimating stock abundance.

Comparison of the results of the review assessments of the Iceland cod with MRI output

0

500

1000

1500

1985 1990 1995 2000

Bio

ma

ss

4-1

4yr

s (

10

00

to

nn

es

)Hilborn

Pope

Gudmundsson

Sullivan

MRI 2000

MRI 2001

Log catchability residuals from 2000 XSA analysis by ICES Working Group

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000

Lo

g c

atc

ha

bili

ty r

es

idu

als

3 years 4 years

5 years 6 years

7 years

Using data supplied by MRI, we have confirmed the appearance of strong retrospective patterns in XSA-estimates of stock abundance which suggest that stock abundances in the late 1990s were overestimated, and hence TACs were set too high.

Initial appraisal of the Problem

Retrospective XSA analyses tuning to combined survey index

-

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1,400,000

1990 1992 1994 1996 1998 2000 2002

Bio

ma

ss 4

+ (

ton

ne

s)

00

99

98

97

96

95

Initial Conclusions

• The retrospective patterns in the XSA results seem to be associated with clear patterns in estimated catchabilities for the commercial fleet and the research surveys, especially the catchabilities in 1997.

• The results of MRI assessments, the Pope review and our analyses to date indicate that the assessment results (i.e. the overestimation of stock size in the late 1990s) are driven by the data available.

• It is the interpretation of the observations that is critical in formulating scientific advice.

Planned Research

There are two questions we are focussing on in our planned research:

1. Could the problem reasonably expect to have been foreseen on the basis of the observed patterns in the data?

2. Can advice be provided that would help to avoid a repeat of such problems in the future?

Planned ResearchTo answer these questions, we are following two main areas

of investigation:

1. Investigating patterns in the data that have given rise to the assessment outputs and how the assessment might be modified to be more robust to such patterns, and

2. Considering the plausibility of observations and assessment results and the concomitant formulation of scientific advice

The overall aim is to provide advice on preventing unnecessary rapid management changes or increased conservation risks due to failure to account fully for uncertainty in the scientific advice for management of Icelandic cod.

Interim results - XSA Assessment

• If only the research survey indices are used in the XSA stock assessment, the retrospective patterns are less prominent, but they are still present.

• Preliminary simulation studies suggest that if the catchability residuals of the surveys are re-sampled to eliminate systematic trends and exceptional values in specific years, then the retrospective patterns suggesting overestimation of stock abundance are no longer present.

• However, the preliminary simulation studies also suggest that it may be the overall trends in catchability residuals, rather than a single exceptional year (e.g. 1997), that led to the overestimation.

Interim results - plausibility of results

• Simulation studies are being used to ask the question for each new observation over the period in question: how reasonable is that observation given our understanding of the biology of the stock, the operation of the fishery and previous assessments?

• Initially, this is being done using a stochastic Schaefer-type production model for two reasons:

– Production models are easier to manipulate that more complicated models such as XSA or time series analyses, and

– Pope’s analysis has shown that this type of model gives a reasonably good fit to the dynamics of age 4+ cod.

• The model is still under development. It has not yet been carefully tuned to match Icelandic cod data as closely as possible; this will be done in the next phase of the analysis

• The graph depicts the distribution of the expected productivity of stocks (the ratio N(t+1)/N(t)), based on assumed stock dynamics for Icelandic Cod

• It shows that annual production in excess of 10% has about a 15% probability, in excess of 20% is possible, but not very likely (about an 8% probability), and in excess of 40% is very rare indeed (about 1.5% probability).

Example output - plausibility of results

Output of the 2000 XSA assessment showing estimated annual production (N(t+1)/N(t)) over the period 1972 to 1999

Example output - plausibility of results

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1972

1973

1974

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

Pro

du

cti

on

((N

t+1

)/N

t)

• This graph depicts the distribution of the actual productivity of stocks, as estimated by the XSA assessment for the period 1972 to 1999

• It indicates that the XSA analysis estimated that annual production of more than 20% happened almost 20% of the time (compared to 8% from the production model analysis)

Example output - plausibility of results

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

0.50

0.8 1 1.2 1.4 1.6 1.8 2

Production ((Nt+1)/Nt)

Fre

qu

en

cy

Spawning Stock Biomasses - Georges BankCOD, HADDOCK and YELLOWTAIL

78 81 84 87 90 93 96 99

YEAR

0

50

100

150

200

ME

TR

IC T

ON

S (

0 00 '

s )

COD HADDOCK YELLOWTAIL

AMENDMENTS#5-#9

Atlantic CodSpringNMFS SurveyDistributions

Year-Round Closed Areas

8

NORTHERN COD

0

400

800

1200

1600

62 67 72 77 82 87 92

0

400

800

1200

1600

2000

RecruitsSSB

Issues Related to Natural Mortality

• Lifespan and literature guide choice of average natural mortality rate

• Small variations without trend not problematic, trends and larger variation can affect annual advice

• Changes in M with age affect estimates of optimal fishing patterns and calculations of reference points

• Simulations guide likely impacts on advice