understanding underwater optical image datasetsoptical modems: 1,000,000 bits/s at 100 meters [farr...
TRANSCRIPT
![Page 1: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/1.jpg)
1 PhD Thesis Defense - J.W. Kaeli, 2013
Computational Strategies for
Understanding Underwater
Optical Image Datasets
Jeffrey W. Kaeli
Advisor:
Hanumant Singh, WHOI
Committee:
John Leonard, MIT
Ramesh Raskar, MIT
Antonio Torralba, MIT
![Page 2: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/2.jpg)
2 PhD Thesis Defense - J.W. Kaeli, 2013
“latency of understanding” in underwater robotics
![Page 3: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/3.jpg)
3 PhD Thesis Defense - J.W. Kaeli, 2013
“latency of understanding” LOU 0 for ROVs
![Page 4: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/4.jpg)
4 PhD Thesis Defense - J.W. Kaeli, 2013
what do we lose by “cutting the cord?”
![Page 5: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/5.jpg)
5 PhD Thesis Defense - J.W. Kaeli, 2013
what can we gain by “cutting the cord?”
![Page 6: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/6.jpg)
6 PhD Thesis Defense - J.W. Kaeli, 2013
AUVs generate large volumes of data
O(1,000s) images
1 image
3 seconds
3600 s
1 hour
O(hours)
x
x
O(10,000,000,000) bits
O(10,000,000) bits
1 image
x
O(1-10) GB
![Page 7: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/7.jpg)
7 PhD Thesis Defense - J.W. Kaeli, 2013
underwater communication channels are limited
Optical Modems:
1,000,000 bits/s at 100 meters [Farr et al. 2010]
Acoustic Modems:
100 - 1,000 bits/s at kilometers
[Frietag et al. 2005]
![Page 8: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/8.jpg)
8 PhD Thesis Defense - J.W. Kaeli, 2013
wavelet-based compression coded with
Set Partitioning in Hierarchical Trees (SPIHT)
smaller packet size optimized for acoustics
fully embedded progressive encoding
throughput of 1 megapixel image / ~15 minutes
acoustic transmission of imagery
[Murphy 2012, PhD Thesis]
![Page 9: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/9.jpg)
9 PhD Thesis Defense - J.W. Kaeli, 2013
manual annotation is time consuming
[Ferrini et al., 2006] [Kaeli 2011] [Futrelle & York, 2012]
![Page 10: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/10.jpg)
10 PhD Thesis Defense - J.W. Kaeli, 2013
automatic classification aids quantitative analysis
habitats [Loomis 2011] [Pizarro et al., 2009]
[Rigby et al., 2010] [Steinberg et al., 2011]
fish [Loomis 2011] [Spampinato et al., 2010]
scallops [Dawkins et al., 2013]
starfish [Clement et al., 2002]
coral [Johnson et al., 2006] [Kaeli et al, 2006]
[Purser et al., 2009] [Soriano et al., 2001]
[Shihavuddin et al., 2013] [Stokes & Deane, 2009]
![Page 11: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/11.jpg)
11 PhD Thesis Defense - J.W. Kaeli, 2013
summary of latencies contributing to overall LOU
O(hours)
? O(days-weeks)
O(hours)
![Page 12: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/12.jpg)
12 PhD Thesis Defense - J.W. Kaeli, 2013
what is the smallest latency with which we can
begin to “understand” the survey environment?
RAW IMAGERY
PRE-PROCESSING
SELECTION
COMPRESSION
TELEMETRY
UNDERSTANDING
{ [Murphy 2012,
PhD Thesis]
Chap. 1,2
Chap. 2,3
Chap. 3
![Page 13: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/13.jpg)
13 PhD Thesis Defense - J.W. Kaeli, 2013
1. Image Correction
2. Computational Strategies
3. Understanding Underwater
Optical Image Datasets
outline of thesis chapters and contributions
![Page 14: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/14.jpg)
14 PhD Thesis Defense - J.W. Kaeli, 2013
1. Image Correction
outline of thesis chapters and contributions
review underwater image formation and broad
range of existing correction techniques
estimate water column properties and unknown
vehicle parameters using multi-sensor fusion
use estimated values to correct images
![Page 15: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/15.jpg)
15 PhD Thesis Defense - J.W. Kaeli, 2013
2. Computational Strategies
outline of thesis chapters and contributions
introduce novel multi-scale image processing
framework for efficient feature computations
discuss design of invariant features
![Page 16: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/16.jpg)
16 PhD Thesis Defense - J.W. Kaeli, 2013
3. Understanding Underwater
Optical Image Datasets
outline of thesis chapters and contributions
propose lightweight keypoint detection and
description scheme for use in scene classification
modify surprise-based navigation summaries for
selecting images to transmit acoustically
demonstrate creation of low-bandwidth semantic
maps via cumulative image telemetry
![Page 17: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/17.jpg)
17 PhD Thesis Defense - J.W. Kaeli, 2013
1. Image Correction
2. Computational Strategies
3. Understanding Underwater
Optical Image Datasets
outline of thesis chapters and contributions
![Page 18: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/18.jpg)
18 PhD Thesis Defense - J.W. Kaeli, 2013
underwater image formation
![Page 19: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/19.jpg)
19 PhD Thesis Defense - J.W. Kaeli, 2013
underwater robotic imaging platform assumptions
ignore gain
no natural light
assume white strobe
neglect spreading
no additive effects
ignore lens effects
![Page 20: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/20.jpg)
20 PhD Thesis Defense - J.W. Kaeli, 2013
log color channel means as a function of altitude
![Page 21: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/21.jpg)
21 PhD Thesis Defense - J.W. Kaeli, 2013
could additional sensors be useful for correction?
SeaBed - Autonomous
Underwater Vehicle
SeaSled – Towed
Camera System
![Page 22: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/22.jpg)
22 PhD Thesis Defense - J.W. Kaeli, 2013
use overlapping imagery to constrain equations
3 equations (RGB), 6 unknowns
![Page 23: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/23.jpg)
23 PhD Thesis Defense - J.W. Kaeli, 2013
use overlapping imagery to constrain equations
3 + 3 = 6 equations, same 6 unknowns!
![Page 24: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/24.jpg)
24 PhD Thesis Defense - J.W. Kaeli, 2013
attenuation coefficient estimates
![Page 25: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/25.jpg)
25 PhD Thesis Defense - J.W. Kaeli, 2013
beam pattern estimate in angular space
![Page 26: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/26.jpg)
26 PhD Thesis Defense - J.W. Kaeli, 2013
artifacts from artificial lighting and attenuation
![Page 27: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/27.jpg)
27 PhD Thesis Defense - J.W. Kaeli, 2013
comparison with our results
raw image
our method frame averaging
white balance (wb) homomorphic + wb
adapt. hist. eq.+ wb
![Page 28: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/28.jpg)
![Page 29: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/29.jpg)
29 PhD Thesis Defense - J.W. Kaeli, 2013
sample corrected imagery
![Page 30: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/30.jpg)
30 PhD Thesis Defense - J.W. Kaeli, 2013
sample corrected imagery
![Page 31: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/31.jpg)
31 PhD Thesis Defense - J.W. Kaeli, 2013
Demonstrated a multi-sensor fusion-based
approach to underwater image correction
Additional sensor information useful for correction,
but makes correction platform-dependent
[Rock et al., unpublished] [Kaeli et al., 2011] [Bryson et al., 2012]
Potential for historical water quality from past data
1. Image Correction
summary, conclusions, future work
![Page 32: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/32.jpg)
32 PhD Thesis Defense - J.W. Kaeli, 2013
1. Image Correction
2. Computational Strategies
3. Understanding Underwater
Optical Image Datasets
outline of thesis chapters and contributions
![Page 33: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/33.jpg)
33 PhD Thesis Defense - J.W. Kaeli, 2013
Convolution is still a major bottleneck in many
multi-scale image processing frameworks such as
fast keypoint detection [Calonder et al., 2010]
Graphics Processing Units (GPUs) have increased
“brute force” computing power [Loomis 2011]
Can we exploit pixel grid geometries that allow us to
substitute adds and bit shifts for multiples while still
approximating a Gaussian? [Viola et al., 2001]
Applications on low-power robotic imaging platforms
![Page 34: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/34.jpg)
34 PhD Thesis Defense - J.W. Kaeli, 2013
the notion of “scale” is synonymous with blurring
Convolution can be thought as locally re-distributing
a signal based on some “kernel” or distribution
Place kernel 𝐾 𝑥 wherever 𝑆 𝑥 is, weight by 𝑆 𝑥 ,
them sum resulting functions
∗ = 𝑆(𝑥) ∗ 𝐾(𝑥) 𝐾(𝑥) 𝑆(𝑥)
![Page 35: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/35.jpg)
35 PhD Thesis Defense - J.W. Kaeli, 2013
the notion of a “scale-space” is a family of images
parameterized by a one-dimensional scale factor [Koenderink 1984] [Babaud et al., 1986]
continuous scale-space [Lindeberg 1994]
discrete pyramids [Burt & Adelson, 1983]
![Page 36: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/36.jpg)
36 PhD Thesis Defense - J.W. Kaeli, 2013
hierarchical discrete correlation (HDC) [Burt 1981]
![Page 37: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/37.jpg)
37 PhD Thesis Defense - J.W. Kaeli, 2013
octagonal pyramid
![Page 38: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/38.jpg)
38 PhD Thesis Defense - J.W. Kaeli, 2013
comparison of effective kernel distributions
𝜎 =1
2 𝜎 = 1 𝜎 = 2 𝜎 = 2 𝜎 = 8 𝜎 = 128
![Page 39: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/39.jpg)
39 PhD Thesis Defense - J.W. Kaeli, 2013
comparison of effective kernel distributions
![Page 40: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/40.jpg)
40 PhD Thesis Defense - J.W. Kaeli, 2013
octagonal pyramid decomposition
computed in 3P adds
and P bit shifts
traditional pyramids
with 5x5 separable
kernels use 3.3P
multiplies, 2.7 adds
for same 2 scale
resolution, 8 times
more operations
![Page 41: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/41.jpg)
41 PhD Thesis Defense - J.W. Kaeli, 2013
invariant features for underwater images
![Page 42: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/42.jpg)
42 PhD Thesis Defense - J.W. Kaeli, 2013
invariant features for underwater images
![Page 43: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/43.jpg)
43 PhD Thesis Defense - J.W. Kaeli, 2013
invariant features for underwater images
![Page 44: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/44.jpg)
44 PhD Thesis Defense - J.W. Kaeli, 2013
what is the smallest latency with which we can
begin to “understand” the survey environment?
RAW IMAGERY
PRE-PROCESSING
SELECTION
COMPRESSION
TELEMETRY
UNDERSTANDING
{ [Murphy 2012,
PhD Thesis]
Chap. 1,2
Chap. 2,3
Chap. 3
![Page 45: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/45.jpg)
45 PhD Thesis Defense - J.W. Kaeli, 2013
2. Computational Strategies
summary, conclusions, future work
introduce novel multi-scale image processing
framework for efficient feature computations
log opponent colors for illumination invariance,
partial correction of attenuation
future work in spatially-varying white balance
[Hsu et al., 2008]
![Page 46: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/46.jpg)
46 PhD Thesis Defense - J.W. Kaeli, 2013
1. Image Correction
2. Computational Strategies
3. Understanding Underwater
Optical Image Datasets
outline of thesis chapters and contributions
![Page 47: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/47.jpg)
47 PhD Thesis Defense - J.W. Kaeli, 2013
create an image “fingerprint” based on histograms
of quantized keypoint descriptors [Sivic & Zisserman 2006]
![Page 48: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/48.jpg)
48 PhD Thesis Defense - J.W. Kaeli, 2013
create an image “fingerprint” based on histograms
of quantized keypoint descriptors [Lowe 2004]
![Page 49: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/49.jpg)
49 PhD Thesis Defense - J.W. Kaeli, 2013
keypoint detection using extrema of the difference
of Gaussian function [Lowe 2004]
![Page 50: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/50.jpg)
50 PhD Thesis Defense - J.W. Kaeli, 2013
keypoint detection using extrema of the difference
of Gaussian function
![Page 51: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/51.jpg)
51 PhD Thesis Defense - J.W. Kaeli, 2013
both methods detect similar keypoints
![Page 52: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/52.jpg)
52 PhD Thesis Defense - J.W. Kaeli, 2013
Quantized Accumulated Histogram of Oriented
Gradients (QuAHOG) around each keypoint
![Page 53: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/53.jpg)
53 PhD Thesis Defense - J.W. Kaeli, 2013
possible QuAHOG patterns
[Ojala et al., 2002]
![Page 54: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/54.jpg)
54 PhD Thesis Defense - J.W. Kaeli, 2013
keypoints detected and described with QuAHOGs
![Page 55: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/55.jpg)
55 PhD Thesis Defense - J.W. Kaeli, 2013
what is the smallest latency with which we can
begin to “understand” the survey environment?
RAW IMAGERY
PRE-PROCESSING
SELECTION
COMPRESSION
TELEMETRY
UNDERSTANDING
{ [Murphy 2012,
PhD Thesis]
Chap. 1,2
Chap. 2,3
Chap. 3
![Page 56: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/56.jpg)
56 PhD Thesis Defense - J.W. Kaeli, 2013
surprise-based online navigation summaries
[Girdhard & Dudek, 2010, 2012]
[Itti & Baldi,2005]
Kullback-Leibler divergence
![Page 57: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/57.jpg)
57 PhD Thesis Defense - J.W. Kaeli, 2013
the threshold for “surprise” grows as the vehicle
experiences more of the world
![Page 58: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/58.jpg)
58 PhD Thesis Defense - J.W. Kaeli, 2013
modified navigation summaries for semantic maps
wait for surprise threshold to stabilize before
selecting first image to transmit (most represented)
once transmitted, a summary image should become
a static member of the summary set
non-summary images are attributed a summary
image to represent them
never remove summary images, merge with nearest
summary image, represent with more represented
![Page 59: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/59.jpg)
59 PhD Thesis Defense - J.W. Kaeli, 2013
selecting which summary image to transmit
![Page 60: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/60.jpg)
semantic mapping via cumulative image telemetry
60 PhD Thesis Defense - J.W. Kaeli, 2013
data collected by SeaBED AUV
in 2003 off Stellwagen Bank
can transmit approximately once
every 300 images
summary size set to 16,
approximately twice as many
images as will be sent
![Page 61: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/61.jpg)
61 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping via cumulative image telemetry
![Page 62: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/62.jpg)
62 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping via cumulative image telemetry
![Page 63: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/63.jpg)
63 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping via cumulative image telemetry
![Page 64: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/64.jpg)
64 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping via cumulative image telemetry
![Page 65: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/65.jpg)
65 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping via cumulative image telemetry
![Page 66: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/66.jpg)
66 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping via cumulative image telemetry
![Page 67: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/67.jpg)
67 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping via cumulative image telemetry
![Page 68: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/68.jpg)
68 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping with heuristic merging
![Page 69: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/69.jpg)
69 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping with heuristic merging
![Page 70: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/70.jpg)
70 PhD Thesis Defense - J.W. Kaeli, 2013
semantic mapping with heuristic merging
![Page 71: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/71.jpg)
71 PhD Thesis Defense - J.W. Kaeli, 2013
post-mission analysis of entire summary set
![Page 72: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/72.jpg)
72 PhD Thesis Defense - J.W. Kaeli, 2013
post-mission analysis of class membership
![Page 73: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/73.jpg)
73 PhD Thesis Defense - J.W. Kaeli, 2013
post-mission analysis of class membership
![Page 74: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/74.jpg)
74 PhD Thesis Defense - J.W. Kaeli, 2013
post-mission analysis of class membership
![Page 75: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/75.jpg)
75 PhD Thesis Defense - J.W. Kaeli, 2013
post-mission analysis of class membership
![Page 76: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/76.jpg)
76 PhD Thesis Defense - J.W. Kaeli, 2013
post-mission analysis of class membership
![Page 77: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/77.jpg)
77 PhD Thesis Defense - J.W. Kaeli, 2013
what is the smallest latency with which we can
begin to “understand” the survey environment?
RAW IMAGERY
PRE-PROCESSING
SELECTION
COMPRESSION
TELEMETRY
UNDERSTANDING
{ [Murphy 2012,
PhD Thesis]
Chap. 1,2
Chap. 2,3
Chap. 3
![Page 78: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/78.jpg)
78 PhD Thesis Defense - J.W. Kaeli, 2013
nDeployments ≥ nRecoveries… LOU ∞
0
2
4
6
8
10
12
14
16
18
20
Deployments
Recoveries
[Kimball 2013]
![Page 79: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/79.jpg)
79 PhD Thesis Defense - J.W. Kaeli, 2013
3. Understanding Underwater
Optical Image Datasets
summary, conclusions, future work
propose lightweight keypoint detection and
description scheme for use in scene classification
modify surprise-based navigation summaries for
selecting images to transmit acoustically
demonstrate creation of low-bandwidth semantic
maps via cumulative image telemetry
![Page 80: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/80.jpg)
80 PhD Thesis Defense - J.W. Kaeli, 2013
summary, conclusions, future work
we have proposed a realistic framework for
reducing the latency of understanding paradigm
in autonomous underwater robotics
future implementations in real AUV missions
more robust keypoint detectors/descriptors
“AUV Pandora” for adaptive mission planning
![Page 81: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/81.jpg)
81 PhD Thesis Defense - J.W. Kaeli, 2013
acknowledgements
![Page 82: Understanding Underwater Optical Image DatasetsOptical Modems: 1,000,000 bits/s at 100 meters [Farr et al. 2010] ... Understanding Underwater Optical Image Datasets outline of thesis](https://reader033.vdocuments.net/reader033/viewer/2022060323/5f0db29c7e708231d43ba35f/html5/thumbnails/82.jpg)
82 PhD Thesis Defense - J.W. Kaeli, 2013
questions?