articulated human detection
DESCRIPTION
Articulated Human Detection . Student: Yao-Sheng Wang Advisor: Prof. Sheng- Jyh Wang. Department of Electronics Engineering National Chiao Tung University. Hsinchu, Taiwan. Vision Lab 2012. 1. Outline. Introduction Related Works Idea Proposed Method Experimental Results - PowerPoint PPT PresentationTRANSCRIPT
Student: Yao-Sheng WangAdvisor: Prof. Sheng- Jyh Wang
ARTICULATED HUMAN DETECTION
1
Department of Electronics Engineering
National Chiao Tung UniversityHsinchu, Taiwan
1Vision Lab 2012
Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
2
Introduction Motivation Challenge Representative Works Potential Problems Target
Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
3
4
Why we care about human detection? We are human beings!
Wide range of applications: Automotive safety Surveillance system
Indoor care Crime alert
Human-Computer Interface … etc.
MOTIVATION
Introduction Motivation Challenge Representative Works Potential Problems Target
Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
5
6
What makes human detection so difficult? Illumination condition Cluttered background Change of viewpoints Occlusion Wearing difference Diversity of human Pose variation
CHALLENGE
7
What makes human detection so difficult? Illumination condition Cluttered background Change of viewpoints Occlusion Wearing difference Diversity of human Pose variation
CHALLENGE
8
What makes human detection so difficult? Illumination condition Cluttered background Change of viewpoints Occlusion Wearing difference Diversity of human Pose variation
CHALLENGE
9
What makes human detection so difficult? Illumination condition Cluttered background Change of viewpoints Occlusion Wearing difference Diversity of human Pose variation
CHALLENGE
10
Progress on “Machine Learning” technologyHandle more general and complicate cases.
Definition: “Articulated Human Detection”.
CHALLENGE
Introduction Motivation Challenge Representative Works Potential Problems Target
Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
11
12
Deformable Part Model Root filter (mask). Part filter (mask). Penalty function.
REPRESENTATIVE WORKS (I)
[P. Felzenszwalb, D. McAllester, and D. Ramanan. A discriminatively trained, multi-scale, deformable part model. In CVPR, 2008.]
13
Pose-let:
REPRESENTATIVE WORKS (II)
[Lubomir Bourdev, Jitendra Malik. Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations. In ICCV, 2009.].
Introduction Motivation Challenge Representative Works Potential Problems Target
Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
14
15
Problems: System complexity increased with the complexity of human poses. More detectors needed.
Exhaustive search. Sliding window method + Image pyramid.
Both problems leads to unacceptable speed for applications in real life.
POTENTIAL PROBLEMS
Introduction Motivation Challenge Representative Works Potential Problems Target
Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
16
17
Target in the thesis: Propose a detection scheme with acceptable detection speed in dealing with highly intra- class variation from the change of pose and viewpoint.
TARGET
Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
18
19
Better features: Cheap to compute and capture crucial information at the same time. Ex: HOG.
Better classifiers: Linear classifiers.
Ex: Adaboost, Linear-SVM and Random-forests.
Better prior knowledge: Ex: Information about ground plane.
RELATED WORKS
20
Cascades: Cascade the part filters to reduce the searching regions.
RELATED WORKS
[P. Felzenszwalb, R. Girshick, D. McAllester. Cascade Object Detection with Deformable Part Models. In CVPR, 2010.]
21
Discard non-promising hypotheses. Class-dependent:
Branch and bound. (CVPR, 2008) Class-independent:
What is an object? (CVPR, 2010) Closure boundary, different appearance or salience.
Segmentation as selective search. (ICCV, 2011)
RELATED WORKS
Start
22
Feature response approximation: Feature approximation in testing step. Feature approximation in training step.
RELATED WORKS
[R. Benenson, M. Mathias, R. Timofte, and L. Van Gool. Pedestrian detection at 100 frames per second. In CVPR, 2012.]
[P. Dollár, S. Belongie, P. Perona. The fastest pedestrian detector in the west. In BMVC, 2010.]
Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
23
24
Recall the memory of the first problem: System complexity increased with the complexity of human poses (include variation of viewpoints).
How can we break the relation between the complexity of system and the one of human poses? Choose stable features or body parts for detection.
IDEA
25
Better prior knowledge:
IDEA
26
Recall the memory of the second problem: Exhaustive search.
“Sliding Window” + “Image Pyramid”. How can we reduce the searching region?
Detect the common feature among these parts.
Use the cumulative characteristic of the feature to handle the variation of scale.
IDEA
27
Common feature Body parts consist of combination of two edge segments.
Cumulative characteristic Edge detector with fixed size + Combination.
IDEA
28
The previous works focus on reducing the searching regions. Specifically against “Exhaustive Search”.
Our method starts from breaking the relation between complexity of system and that of poses. Then, use the common feature and cumulative characteristic to cut down the searching space.
COMPARISON
Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
29
30
Fast Part Detection
Part Combinati
on
Combination
Refinement
SYSTEM BLOCK
Bottom-up system:
31
Fast Part Detection
Part Combinati
on
Combination
Refinement
SYSTEM BLOCK
Bottom-up system:
32
Steps: Detection of edge candidates. Production of part candidates. Refinement of part candidates.
FAST PART DETECTION
33
Detection and combination of segments (9 orientations).
DETECTION OF PART CANDIDATES
34
Constraints on combination of edges. Orientation, length ratio and color symmetry.
PRODUCTION OF PART CANDIDATES
Neighbor orientation consideration
35
HOG feature + Random forest training
REFINEMENT OF PART CANDIDATES
Feature = [Length Orientation HOG_features]
feature134
feature33
feature2
? ?
feature400
36
Fast Part Detection
Part Combinati
on
Combination
Refinement
SYSTEM BLOCK
Bottom-up system:
37
Problem: No information about the classes of the limbs
due to the low resolution of images or variation from hand gestures or appearance of shoes...etc.
Need another step to refine the combinations. What information left?
Head-shoulder or head-torso.
PART COMBINATION
38
Any possibility for us to estimate the position and orientation of head-torso based on the architecture of current combinations?
PART COMBINATION
39
Problem: How to select body parts belong to specific human from lots of part candidates?
Too much possibilities for exhaustive search.
PART COMBINATION
40
Clues for reducing the number of possible combinations. Center distance, length ration or width ratio between two parts.
Combination with the number of parts more than four.
PART COMBINATION
41
Conclusion for the clues mentioned in the previous slide. Too complicate to combine the parts for the whole body.
Start from low-level combination of parts to reveal the benefits of physical constraints.
Break the problems into two levels. Low-level combination. High-level combination.
PART COMBINATION
42
How far can we reach for low-level combination? 4-parts combination = lower body.
LOW-LEVEL COMBINATION
43
False alarm exists.
Joints relative position + Random Forest
LOW-LEVEL COMBINATION
feature134
feature33
feature2
? ?
feature400
44
Combination between the arms, legs, lower bodies and uncombined single parts from the low-level combination step. Upper bound of the number of combination:
HIGH-LEVEL COMBINATION
45
Fast Part Detection
Part Combinati
on
Combination
Refinement
SYSTEM BLOCK
Bottom-up system:
46
Pose prediction. Detection with DPM detector.
COMBINATION REFINEMENT
47
Feature: Relative size ratio and positions between low-level combinations and architecture of each low-level combination.
Random Forest.
POSE PREDICTION
48
Use DPM detector to cover the intra-class variation.
Model:
DETECTION WITH DPM DETECTOR
49
Much stronger than information of limbs. Head-shoulder to head-torso. Start from head-torso to combine limbs back.
USAGE OF HEAD-SHOULDER INFORMATION
50
SYSTEM ILLUSTRATION
Edge Candidates
Part Candidates
Low Level Part Combine
Pose PredictionHead-Torso Detection
High Level Part Combine
Part Detector
Parts
Low Level Combination
Result ofDetection
High Level Combination
Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
51
Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
52
Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference
OUTLINE
53