vision system for wing beat analysis of bats in the wild 1 boston university department of computer...
TRANSCRIPT
Vision System for Wing Beat Analysis of Bats in the Wild
1Boston University Department of Computer Science2Boston University Department of Biology
Mikhail Breslav1, Nathan W. Fuller2, and Margrit Betke1
ComputerScience
Motivation• Behavior• Aerial Vehicles
Past Work
• Study Kinematics (Hubel. 2012)
– Wind Tunnel– High Resolution Cameras
• 3D Tracking (Wu. 2009)
– Outdoor environment – Model Bats as Points
• Behavior and Trajectory Analysis (Theriault. 2010, Fisher. 2010)
Goal• Estimate Wing Beat Frequencies– Potential to improve tracking
TWing Beat Frequency: 1/T
Hubel et al. 2012
Challenging Data• Unpredictable Motions• Relatively Low Resolution• In FOV for a Short Time
Segmentation and Tracking• Foreground / Background Estimation• Kalman Filter
Shape-time Signals• Output of Tracker• Define “Shape”
Prototype Shapes
• Assumption– There are shapes that uniquely identify 3D
poses for a given camera
• Example
• Currently chosen manually
“up” “down” “neutral”
Main Idea• A prototype shape is equal to a 3D
pose• Repeating prototype shapes in a
shape-time signal
Estimate Wing Beat
Shape Comparison• Shape Distance– Shape Context Descriptor (Belongie et al. 2002)• Invariant to translation, scale, and optionally rotation
– Hungarian Algorithm• Establish Correspondences
• Estimate Wing Relative to Body with feature W
Shape Similarity Scores• Use Shape Distance and Ratio W to assign
similarity score– Also consider the ‘none’ hypothesis
‘None’
.24 .16 .43 .17
“up” “down” “neutral”
Process Shape-Time Signal
• Find confident matches to prototype shapes
“up”
Process Shape-Time Signal
“down”
“neutral”
Time Axis
Fast Fourier Transform
Time Axis
“up”
“down”
“neutral”
FFT
Fast Fourier Transform
Periodicity Estimate of 9.76 Hz
Experimental Results• 20 Bats • Both Automatic and Manual estimates
Discussion• Reasonable Estimates– Deviates from manual annotations by 1.3 Hz on
average, standard deviation 1.8 Hz– Falls within 10-15 Hz as reported in biology
literature (Foehring. 1984)
• Main Contribution– System for using shapes to estimate wing beat– First to do this for bats in the wild • Vision based system
Future work
• Choosing prototype shapes – Automatically and Intelligently
• Understand the mapping between 2D shapes and 3D poses for a given model– Generalize across datasets
• Try more robust shape comparison measures
Thank You for your Attention!
Questions?
Holding a bat!