© devi parikh 2008 localization and segmentation of 2d high capacity color barcodes gavin jancke...
TRANSCRIPT
© Devi Parikh 2008
Localization and Segmentation of
2D High Capacity Color Barcodes
Gavin Jancke
Microsoft Research, Redmond
Devi Parikh
Carnegie Mellon University
© Devi Parikh 2008
Motivation
UPC Barcode
QR Code Datamatrix
© Devi Parikh 2008
HCCB
Microsoft’s High Capacity Color Barcode
© Devi Parikh 2008
Application Uniquely identifying commercial audiovisual
works such as motion pictures, video games, broadcasts, digital video recordings and other media
© Devi Parikh 2008
GoalLocate and Segment the barcode from consumer
images
© Devi Parikh 2008
Overview
Design specifications of Microsoft’s HCCB
Approach
Localization
Segmentation
Progressive Strategy
Results
Conclusions
© Devi Parikh 2008
Microsoft’s HCCB
4 or 8 colors
Triangles
String of colors
palette
© Devi Parikh 2008
Microsoft’s HCCB
© Devi Parikh 2008
Microsoft’s HCCB
© Devi Parikh 2008
Microsoft’s HCCB
© Devi Parikh 2008
Microsoft’s HCCB
R rows
S symbols per rowS = (r+1)*R
Aspect ratio: r
© Devi Parikh 2008
Approach
Thresholding
Orientation prediction
Corner localization
Row localization
Symbol localization
Color assignments
Barcode localization
Barcode segmentation
point inside the barcode is known
© Devi Parikh 2008
Localization: Thresholding
Identify thick white band and row separators
Normalization
Adaptive
© Devi Parikh 2008
Localization: Orientation
orientation orientation
dist
ance
-90 900sum
mat
ion
© Devi Parikh 2008
Localization: Corners Rough estimates
whiteness mask non-texture mask combined mask
© Devi Parikh 2008
Localization: Corners Gradient based refinement
© Devi Parikh 2008
Localization: Corners Line based refinement
© Devi Parikh 2008
Segmentation: Rows
Summation
Flip?
© Devi Parikh 2008
Segmentation: Symbols
S E
Local search
Number of symbols per row
q(S,E) = q(samples|S,E)
© Devi Parikh 2008
Segmentation: Colors
Palette
© Devi Parikh 2008
Segmentation results given accurate localization Satisfactory
Corner localization Unsatisfactory
No one strategy works well on all images However (1) Errors of different strategies are
complementary (2) Results are verifiable with decoder in the
loop!
Observations
© Devi Parikh 2008
Progressive strategy Hence – progressive strategy!
Similar to ensemble of weak classifiers Or hypothesize-and-test
Multiple strategies: Rough + gradient + line, or rough + line, or
rough + gradient, or rough alone Different values of threshold during rough
corner detection Total 12
Order of strategies
© Devi Parikh 2008
Results
Dataset of 500 images
Performance metric: % barcodes successfully decoded
Decoder model: Barcode successfully decoded if 80% of symbols are correctly identified
© Devi Parikh 2008
Results
Allows for explicit trade-off between accuracy and computational time
© Devi Parikh 2008
Results
© Devi Parikh 2008
Results
© Devi Parikh 2008
Results
© Devi Parikh 2008
Results
© Devi Parikh 2008
Results
© Devi Parikh 2008
Results
© Devi Parikh 2008
Results
© Devi Parikh 2008
Results
© Devi Parikh 2008
Results
© Devi Parikh 2008
Results
© Devi Parikh 2008
Conclusions 2D High Capacity Color Barcode (HCCB)
Successful localization and segmentation of HCCB from consumer images
Varying densities, aspect ratios, lighting, color balance, image quality, etc.
Simple computer vision and image processing techniques
Progressive strategy
© Devi Parikh 2008
Acknowledgements
Microsoft Research Larry Zitnick Andy Wilson Zhengyou Zhang
Carnegie Mellon University Advisor: Tsuhan Chen
© Devi Parikh 2008
Thank you!