observer’s associate

Scientific Fishery Systems, Inc

1

Observer’s Associate

A consistent, unbiased system using machine

vision and fish morphometrics to identify

species

From

Scientific Fishery Systems, Inc.P.O. Box 242065

Anchorage, AK 99524

907.563.3474

Dr. Eric O. Rogers


2

Observer’s Associate Team

• Principle Investigator - Pat Simpson - SciFish

• Lead Scientist Eric O. Rogers, PhD (Physics) - SciFish

• Luke Jadamec, Fisheries Observer Trainer

• Joe Imlach PE, PhD (ME) Imlach Consulting

• Chris Bublitz, UAF Fisheries Industrial Technology Center


3

Issues identified by SciFish

• Increasing pressure on resource

• Increasing complexity of new legislation

• Possible environmental changes affecting fishery in unknown ways

• Appropriately harvesting and managing the fishery are increasingly difficult tasks

=> Need the best data possible <=


4

Current Sources of Data

• AFSC Survey Trawls– Practical limits to time and scope

• Observer’s Reports– Most effective means of

monitoring CPUE– Statistically small sample– Potentially biased by factors

outside observer’s and vessel operators control

– Of questionable value in legal action due to statistical nature of data


5

SciFish’s Proposal• Using funding form the NSF build

and test an automated onboard fish cataloging system using COTS Hardware and Software that will:– Assist commercial fishery observers

with their monitoring and assessment tasks at sea

– Provide detailed unbiased species counts to manage the Community Development Quota (CDQ) program in Western Alaska

– Provide new detailed information on the ecological health of each species to assist in fisheries management

– Provide detailed information on fish morphometrics that will be of value to researchers in several academic areas, such as fish population studies and fish evolution


6

Key Concepts

• COTS hardware and software

• Candle the fish to separate from background

• Machine Vision and Morphometrics

• Neural Net

• Sample all the fish

• System scales - can add CPU’s for faster processing and add metrics and/or color for greater accuracy


7

Observer’s Associate Benefits

• More and better data means fewer surprises for managers and skippers

• A healthier fishery through management based upon more complete knowledge

• Sample entire catch, no extrapolation• Fair and impartial catch statistics - a

level playing field• Easy to identify and reward “clean”

Vs “dirty” boats• Brings in non-traditional funds for

fisheries research (NSF $)• Fringe Benefit => Provides length,

width, etc. for each fish in addition to species


8

Observer’s Associate Mechanical Design

Transparent conveyor

Light table

Light reflector

Camera(s)

Image processor in watertight compartment

Heat exchanger

Belt cleaner


9

Observer’s Associate Logic Flow

ImageCapture

BoundaryDetection

MeasureFish

IdentifySpecies

DataStorage

Fish

Imag

e

FishMetrics

Fish

Species

Fish O

utline

Fish Im

age

Fish M

etrics


10

Flatfish Features Used by People


11

Typical Flatfish Features Used by Machine Vision

Body Width Standard LengthTail Length Standard LengthTail Fork Length or Max width to tip for rounded tails Standard LengthBody Width Standard Length(Total Width Body Width) / Standard Length(Ellipse {standard length and body width} - body perimeter) Standard Length“Fin” Perimeter (Total Perimeter – Body Perimeter) Standard Length (Ellipse Area – Body Area) (Standard Length * Body Depth)Fin Area / (Standard Length)2


12

Concept Test

• Scan Pictures from Northeast Pacific Flatfishes Book– Scale to meter stick in picture

– Extract measurements

• Reduce measurements to independent metrics– Principle component analysis

• Train Neural Net• Create 100 fish / species by adding

various percentages of white noise• Test classifier with “white noise”

fish


13

Normalized Machine Vision Flatfish Metrics

W1 W2 W3 W4 W5 W6 W7 W8Arrowtooth Flounder -3.79 0.06 -0.28 0.33 -0.58 0.17 -0.33 -0.69Greenland Turbot -2.95 -0.06 -1.31 -0.22 0.47 -0.48 -0.33 0.48Pacific Halibut -1.89 -0.99 -1.37 0.00 -0.55 -0.39 0.52 0.76Rex Sole -1.75 0.26 1.06 -0.26 0.80 0.37 0.62 0.82Dover Sole -1.18 -1.98 0.70 0.44 0.16 0.56 -0.58 0.16Deepsea Sole -0.61 -0.49 1.70 -0.34 -0.17 -1.02 -0.45 -0.43Sand Sole -0.56 1.11 0.06 0.18 0.25 0.18 -0.44 0.67Petrale Sole -0.51 0.40 -0.10 0.14 0.43 -0.22 -0.16 -0.91Kamchatka Flounder -0.17 1.27 -0.34 0.10 -0.33 0.16 0.27 0.12English Sole 0.00 1.00 -0.87 -0.03 0.11 0.20 1.21 -0.65Bering Flounder 0.51 1.93 0.23 -0.49 -0.25 0.21 -0.67 -0.35Flathead Sole 0.93 0.87 0.51 -0.09 -0.18 0.73 -0.36 -0.40Rock Sole 1.15 -1.58 1.10 0.24 0.23 -0.25 0.59 -0.88Butter Sole 1.56 -0.55 1.46 -0.05 -0.58 0.08 0.39 1.27Yellowfin Sole 2.31 0.48 -0.20 0.06 -0.09 -0.21 0.74 -0.32Starry Flounder 2.74 -2.77 -1.58 -0.45 0.04 0.43 -0.39 -0.25Alaskan Plaice 4.19 1.05 -0.78 0.46 0.23 -0.51 -0.64 0.60

TailLength

Tail ForkLength

BodyDepth

FinDepth

El-bPerimeter

"Fin"Perimeter

El-bArea

FinArea

Pacific Halibut 0.9 0.7 0.8 0.9 1.0 1.1 0.5 0.5Alaskan Plaice 1.6 1.2 1.3 1.5 1.5 1.5 1.3 1.6Greenland Turbot 0.9 0.0 0.8 0.9 2.0 0.8 0.5 0.5Arrowtooth Flounder 0.7 0.0 0.8 0.4 1.2 0.7 0.8 0.4Dover Sole 1.0 1.2 0.9 0.8 -0.4 0.7 0.5 0.9Rex Sole 0.9 1.3 0.7 0.9 1.9 0.7 0.9 1.0Yellowfin Sole 1.1 1.3 1.2 1.3 1.0 1.3 1.4 1.3Flathead Sole 1.2 1.2 1.1 1.0 1.1 1.0 1.5 1.1Rock Sole 0.8 1.7 1.2 1.2 -0.4 1.0 1.0 1.4Butter Sole 1.2 2.4 1.0 1.0 0.9 1.1 1.0 1.3Starry Flounder 1.3 1.4 1.2 1.7 -1.6 1.2 0.7 0.9Kamchatka Flounder 1.1 0.8 0.9 0.9 1.9 1.1 1.3 0.9Deepsea Sole 0.8 1.5 1.2 0.7 1.3 0.8 0.8 1.1Petrale Sole 0.9 0.4 1.0 1.0 1.2 0.9 1.1 1.0English Sole 0.9 0.6 0.9 1.2 1.4 1.2 1.4 1.0Sand Sole 1.1 0.8 0.9 0.9 2.0 0.9 1.0 1.0Bering Flounder 1.2 0.8 1.1 0.9 2.2 1.0 1.5 1.0

Metrics after reduction to Principle Component Vectors


14

0

1 0

2 0

3 0

4 0

5 0

6 0

7 0

8 0

9 0

1 0 0

0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 0

W h it e N o is e ( % )

Misc

lassif

ied (#

/1700

sam

ples

)

F a ls e P o s i t i v e F a ls e N e g a t i v e

F ig u r e 3 . Y e l lo w f in S o le M is c la s s i f ic a t io n P e r c e n t a g e V s . W h it e N o is e

0

1 0

2 0

3 0

4 0

5 0

6 0

7 0

8 0

9 0

1 0 0

0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 0

W h it e N o is e ( % )

Mis

clas

sifie

d (#

/170

0 sa

mpl

es)

F a ls e P o s it iv e F a ls e N e g a t iv e

F ig u r e 4 . R o c k S o le M is c la s s i f ic a t io n P e r c e n t a g e V s . W h i t e N o is e

Neural Net Classification Results


15

Observer’s Tasks

• Identify Species that Observer’s Associate does not

• Quality Control

• Ensure Appropriate Sampling

• Operate the Observer’s Associate

• Ensure data integrity and file reports


16

Plan

• Assemble Advisory Panel

• Apply for ASTF Bridge Grant

• Build “Proof of Concept” Prototype

• Train and Test Prototype

• Apply for NSF Phase II Grant

• Build true prototype

• Test for volume onshore

• Test for suitability at sea

• Initial implementation in the Yellowfin Sole fishery


17

Advisors PanelComposition

• Regulators

• Conservationists

• Fisheries Scientists

• CDQ Groups

• Fishermen

• Owners

• Fisheries Consultants


18

Advisory Panel Questions

• Are the issues identified by SciFish of Concern to the industry?

• Is the technology presented a viable solution?

• Are the other, more appropriate solutions to the problems?

• What is the best way to implement this solution?

• Design Changes?• Are there other applications to add

value to the system?• Number of classes for fish Vs

accuracy of classification, Vs throughput of fish Vs cost

observer’s associate

Documents

commercial fishery observers

skippersa healthier

fish morphometrics

fish population studies

pbrush observers

max width

unbiased species

fisheries research nsf