sign classification boosted cascade of classifiers using university of southern california thang...
TRANSCRIPT
Sign Classification
Boosted Cascade of Classifiers
using
University of Southern California
Thang Dinh [email protected] Kim [email protected] Zhang [email protected] Lin [email protected]
Computer Vision group:
Jan 2007
2. Problem statement
Content
1. Sign classification: Applications and Challenges
3. Why Boosted Cascade of Classifiers ?
4. Sign detection and classification with Cascade5. Experimental Results
Jan 2007
Sign classification: Applications & Challenges
Applications
Sign language
……..
Virtual Reality
Robot Control
Sign Classification
Jan 2007
Sign classification: Applications & Challenges
Challenges
1. Detection Rate
2. Computation Time
An interpreting system that interprets wrongly, a robot that always misunderstands commands… can not be employed.
Robots that need 30s to understand a command telling them to do something immediately, Systems that need 30s to interpret each word of speakers… will not be employed either.
need a Fast and Robust system
Jan 2007
2. Problem statement
Content
1. Sign classification: Applications and Challenges
3. Why Boosted Cascade of Classifiers ?
4. Sign detection and classification with Cascade5. Experimental Results
Jan 2007
Problem statement
Human Gesture:
• Dynamic Gesture: requires motion of hand, body…• Static Gesture: static pose of hand, body…
• Finger Spelling
(ASL System)
Jan 2007
Problem statement
Implemented system:
Challenges:
• Detection rate, Computation time• Many similar signs
….
Jan 2007
2. Problem statement
Content
1. Sign classification: Applications and Challenges
3. Why Boosted Cascade of Classifiers ?
4. Sign detection and classification with Cascade5. Experimental Results
Jan 2007
Why Boosted Cascade of Classifiers
• Cascade of boosted classifiers with Haar wavelet features (Viola and Jones) is currently ‘The state of the art’ in face detection.
• Cascade was brought to field of hand detection by Eng-Jon with impressive results.
Jan 2007
Why Boosted Cascade of Classifiers
Face detection system of Viola and Jones is:
• 15 times faster than Rowley’s (double layer Neural Network)• 600 times faster than Schneiderman-Kanade’s (Statistics)
Cascade of boosted classifiers seems to be a good approach for the Sign classification problem.
Jan 2007
2. Problem statement
Content
1. Sign classification: Applications and Challenges
3. Why Boosted Cascade of Classifiers ?
4. Sign detection and classification with Cascade5. Experimental Results
Jan 2007
Haar wavelet features
• Haar wavelet features (Papageorgiou )
• Feature set was enlarged by Lien Hart to 45º rotated.
• We proposed Double-L for Sign detection
Jan 2007
Integral Image – Fast calculation of Haar features
• There are millions of features that need processing and each feature itself needs time to be calculated makes the training process time-consuming
• Viola and Jones proposed Integral Image in order to reduce the feature calculation complexity, which results in less computation time
Definition of intergral image at point (x, y)
Sum(D) = 1 + 4 – (2 + 3)
Then:
Jan 2007
Adaboost
• Proposed by Freund and Schapire
• Combine many ‘weak’ hypothesis to form a ‘strong’ one
• ‘weak’ classifier ht only need to be better than chance
Jan 2007
Adaboost with Haar wavelet features
Advantages:
• Adaboost adjusts adaptively the errors of the weak hypotheses
• Require large amount of training samples
• Weak Hypothesis learnt from this algorithm can run very fast because of simple calculation of Haar feature, which speeds up the whole system
Disadvantages:
• The training process is rather time-consuming because the algorithm needs to check through millions of features extracted from thousands of samples.
Jan 2007
Cascade of classifiers
• Cascade of boosted classifiers is a tree of classifiers where classifier lying at each stage is better than the last• Only those input patterns having passed through all the layers are considered objects
Simple backgrounds can be easily rejected by one-feature classifier
Jan 2007
Sign Classifiers
• Each Sign Detectors Di is a cascade of boosted classifier evaluating an input image to give out value vi which is then compared to the threshold of it
• 24 Sign Detectors are combined to form a Sign Classifier
C = signi | Di = signi and g(Di)=max{g(Dj)|j=1, 2,… 24}
Where g(Di) = |vi – i|
Jan 2007
Sign Classifiers
Jan 2007
2. Problem statement
Content
1. Sign classification: Applications and Challenges
3. Why Boosted Cascade of Classifiers ?
4. Sign detection and classification with Cascade5. Experimental Results
Jan 2007
Experimental Results
Samples Collection
• Collect raw image
• Find the ‘key’ point (which make it different from others)• Cut and Align the image (base on ‘key’ point)
Image B - Raw
Image B - Focus
Image B - Aligned
• Aligned images are finally made greyscale
Jan 2007
Experimental Results
Training Process
• First stage:
• Positive: Thousands of aligned sign images• Negative: Background only – Buildings, paintings, trees, other parts of human body… (also non-sign)
• Second stage:
• Positive: also thousands of aligned sign images• Negative: aligned images of other signs
Fast eliminate simple background
Distinguish each sign from others
Jan 2007
Experimental Results
Results
1. Detectors
• We have trained 24 detectors for 24 signs• Average number of stages: 14
• Total features: approx 123
• Detection rate: 90% - 100%
• Test Images were divided into 2 groups
• Simple background
• Complex background
DR = 100%
FA = 0%
DR = …
FA = …
Jan 2007
Experimental Results
Results
1. Detectors (cont)
Hit rate diagram
Jan 2007
Experimental Results
Results
1. Detectors (cont)
False alarm
High FA rate due to similarity between signs
Jan 2007
Experimental Results
Results
2. Classifiers
Test Images: 600
Correct rate: 83%
Jan 2007
Thank you
Thank you